Consumer hardware is perfectly capable of running much larger models too if you give it enough RAM, and is only held back by RAM being expensive.

And average users having, at most, 8GB of RAM, with 2GB being eaten by Windows and another 2, minimum, by Chrome. Remember, average user is the person who doesn't bother with latest games and doesn't run linux, they only do some office stuff, watch videos and browse social media.

But prices for RAM will eventually come down again and hardware in general will keep becoming more capable.

At best, I expect RAM prices to drop back to their sane price somewhere in 2029-30.

Also, you may not have noticed, but for average users (people that mostly just browse social media and watch videos), mid-spec computers from 2013 are almost perfectly serviceable today - just swap the HDD for a SSD and you're golden. Smartphones have also stopped getting significant hardware upgrades, despite shittier software trying to force obsolescence on older, mid-spec models.

Why would hardware stop improving exactly now?

Because current fabrication process of chips seems to be very close to limits imposed by the laws of physics. The 1nm fab process that TSMC might manage to pull by next year is possibly the literal limit of MOSFET. Unless someone comes up with a different fabrication process that can reach atomic precision, or somehow get more power efficiency from the same area/volume of transistors, or some wonder material, there's no reason to expect hardware to keep improving at the same pace it did decades ago. "AI will discover-" It won't. There is nothing indicating it might, much less will. I'd love to be proven wrong, but I seriously doubt AI will magic out some new way to get Moore's law back on track

Accordingly, the AI models shipped with consumer devices will keep becoming more powerful and online models will be needed less and less for “average person” tasks.

I seriously doubt that for a single reason: money. Why let the user have the model when you can rent it off to them? Or, even worse, the model runs locally, but you have to pay a fucking subscription in order to use it because fuck you. "You will own nothing and be happy about it"