If done correctly it could actually be useful. It could read the input text, select the numbers above, and feed those numbers to the sum command.
But no, the language model is told to solve a mathematical equation with no assistance. Of course it's going to fail.