I doubt OpenAI will fundamentally change how GPT works, but you never know. Possibly the next gen of LLM's will learn from the failings of this generation and be fundamentally constructed better.

Not saying I don't realise the cost, either. Better LLM's will arguably have a much worse outcome for humanity. I'm just talking here about personal ways to get some use out of them. I actually started writing an article about how this might help in brainstorming sketches, but I'm probably not going to ever finish / publish it.