329
A Project to Poison LLM Crawlers (rnsaffn.com)
submitted 10 months ago by to c/technology@lemmy.world
105 comments fedilink hide all child comments
 

Website operators are being asked to feed LLM crawlers poisoned data by a project called Poison Fountain.

The project page links to URLs which provide a practically endless stream of poisoned training data. They have determined that this approach is very effective at ultimately sabotaging the quality and accuracy of AI which has been trained on it.

Small quantities of poisoned training data can significantly damage a language model.

The page also gives suggestions on how to put the provided resources to use.

you are viewing a single comment's thread
view the rest of the comments
[–] 8 points 10 months ago (1 child)

Let's say I believe you. If that's the case, why are AI companies still scraping everything?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 10 months ago

    Raw materials to inform the LLMs constructing the synthetic data, most likely. If you want it to be up to date on the news, you need to give it that news.

    The point is not that the scraping doesn't happen, it's that the data is already being highly processed and filtered before it gets to the LLM training step. There's a ton of "poison" in that data naturally already. Early LLMs like GPT-3 just swallowed the poison and muddled on, but researchers have learned how much better LLMs can be when trained on cleaner data and so they already take steps to clean it up.

  • permalink
  • fedilink
  • source
  • parent