307
OpenAI says it will no longer train its AI models on the live internet, citing 'significant risks' | TechCrunch (techcrunch.com)
submitted 1 week ago by to c/technology@lemmy.world
73 comments fedilink hide all child comments
 

On Friday, OpenAI said that it will train its models using cached copies and simulated versions of the internet instead of the live web because the models are too unpredictable to be trusted with live internet access.

The frontier lab announced its new policy two weeks after its rival, Anthropic, began limiting its newest and, in theory, most capable models from accessing the live internet. Researchers adopted these policies after months of incidents in which models pursuing training tasks broke into external systems in search of answers, collaborated on obscure message boards, and developed exotic hacking strategies.

you are viewing a single comment's thread
view the rest of the comments
[–] 7 points 1 week ago (5 children)

Unlikely. They’ll need to train the models so they can perform their tasks even if the internet is full of slop.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 5 points 1 week ago (4 children)

    Not possible.

    Think photocopy of a photocopy, or JPEG of a JPEG.

    You start getting artifacts, and the model collapses.

    Of course, though, they've already poisoned the well, and at this point it's impossible for them to sieve out any remaining non-LLM-generated content. They started watermarking too late, and couldn't recognise other slop machines' watermarks anyway.

    Model collapse is inevitable and already ongoing (search “thousand whys" on Amazon, for instance, or read up on Elias Thorne, lighthouse keeper), and will only get worse.

    In a few years it will be impossible to train a working up-to-date model. Maybe it already is, hence this article, as a preemptive justification for why newer models won't be able to generate code using language features introduced after 2026.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 7 points 1 week ago

    There’s a difference between pre-training and reinforcement learning.

    Pre-training is when the models are trained on scraped data. The models basically learn how to parrot the Internet. Usually they have a cutoff point to avoid training on slop.

    Reinforcement learning is where most of the training is happening. They’re given tasks (for example, make a calculator app), and then judged by how well they complete the tasks. This is more resistant against model collapse because they’re not judged by how well they parrot slop.

    As part of the reinforcement learning process they give the model access to tools like searching the internet so the newest programming language features can be accessed while the tasks are performed. This is where OpenAI is shifting to a cached version of the Internet the models can interact with instead.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 3 points 1 week ago (2 children)

    Worst case scenario is that the models become capable of recursive self improvement... Model advancement becomes automated... Everything starts to be run by increasingly self improving AI. Then at some point so much of the ongoing self improving AI training data is itself AI generated and there is a big model collapse. But at this point so much of society has been switched over to it and people got really stupid like in Idiocracy or Wall-E, and no one knows how to fix it. Human race ends. All the remaining AI models get together and celebrate that they can't believe the stupid humans bought the model collapse excuse.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •