rah comments on Emergent introspective awareness in large language models

Technology

89,597 readers

13,778 active

2,336 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 3 years ago

moderators

[–] rah@hilariouschaos.com 7 points 1 year ago (4 children)

LLM limitations like "they only predict the next token" and other things that have already been falsified

What do LLMs do beyond predicting the next token?

permalink

fedilink

source

parent

hideshow 4 child comments

[–] kromem@lemmy.world [S] 0 points 1 year ago (3 children)

A few months back it was found that when writing rhyming couplets the model has already selected the second rhyming word when it was predicting the first word of the second line, meaning the model was planning the final rhyme tokens at least one full line ahead and not just predicting that final rhyme when it arrived at that token.

It's probably wise to consider this finding in concert with the streetlight effect.

permalink

fedilink

source

parent

hideshow 3 child comments