Yeah, but LLM innovation now is not in more clever architectures, but rather larger and larger models with more training data.

I don't hate the existence of LLMs but rather how they're being shoehorned everywhere and how much power is being spent for just a little bit better results.