322
Ladies and gentleman, we have reached peak Agentic AI Coding - Goblin instructions in OpenAI's Codex system prompt (lemmy.ca)
submitted 5 months ago by to c/programming@programming.dev
34 comments fedilink hide all child comments
 

In case you missed it, ChatGPT 5.1 had a tendency to talk about "goblins" in its responses. Supposedly this was a result of training a "nerdy" personality, but it bled into the model as a whole. Because the training run for the latest model already had this flaw, they had to add specific instructions to the system prompt for their Codex coding tool to avoid this behaviour.

Here's the full prompt from their github. In fact, they repeated the goblin instructions twice, cos you know that will definitely fix it. It's an interesting read if you consider each one of these instructions were meant to prevent some undesired behaviour: https://paste.sh/Iev3HtMe#JZ4dw_CkvJcpVmjjoy7WZnSn

More info here: https://news.northeastern.edu/2026/05/06/chatgpt-goblins-problem-ai-behavior/

OpenAI's own blog post casually explaining why they couldn't predict that their state of the art model would obsess about goblins: https://openai.com/index/where-the-goblins-came-from/

A snippet from the system prompt of OpenAI's Codex coding tool, where they instruct the tool to "Never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures"
you are viewing a single comment's thread
view the rest of the comments
[–] -4 points 5 months ago*

OpenAI should be able to re-train its poorly trained model. But of course it can't, that would take months, maybe years of datacenter time.

Why speak on subjects that you clearly have no knowledge or experience with?

Training is checkpointed and can be continued without retraining. Finetuning a model that has already been trained is a different process from training, and does not take months or years of datacenter time.

But Microsoft can modify the Windows 11 source code. Or at least they used to be able to, before AI.

Huh? It takes way more time and effort to develop new features and changes for software like Windows.

  • permalink
  • fedilink
  • source
  • parent