Arguably, thinking is extreme pattern matching. And, they can make original things that were definitely not in their training data.
The problem seems to be more about alignment. They're rewarded for generating human-looking text, nothing more, and we have no obvious alternative to training them that way. So, of course they're a bit imprecise at any other task.