you are viewing a single comment's thread
view the rest of the comments
[–] 44 points 3 months ago (3 children)

You can only stave off model collapse so long.

It’s only a matter of time before most of the data going in to LLMs came out of one too.

Like I’m not even good at math or anything, much less computer science, but that seems obvious and inevitable to me.

Maybe someone smarter than me can chime in.

  • source
  • hideshow 3 child comments
  • [–] 21 points 3 months ago* (last edited 3 months ago) (2 children)

    It's kind of sort of already happening. Emdashes are fragments of it, a token saving strategy introduced by already trained on old books in their data sources. Model collapse is inevitable.

  • source
  • parent
  • hideshow 2 child comments