No, controlling the behavior by providing a hand-tuned list of no-nos shows that we have no idea how to make an AI stay on task.
This is how you make an AI stay on task. This is how literally anything with any semblance of uncertainty is programmed - you provide guide rails to keep the behaviour confined to whatever you want to limit it to. It's just we don't normally used plain English to do it.
Even koopa trooper in super Mario Bros has guide rails dictating the limits of its behaviour. They turn when they reach blocks. Red ones turn when they reach a ledge. That's for something that literally just moves across a screen horizontally. ChatGPT is trained to output text for a nearly unlimited number of topics - it's insane to think that you wouldn't need a fair number of guide rails.
