They probably won't. Natural neural networks don't fit all that stuff into their working memory either. But some people can imagine the full thing in all detail while they create it in the physical world.
I guess, the AI gets some abstract description (probably not human readable text; but who knows) of the objects in the scene and does with its artificial neural network what those people do with their natural neural network - at 60 FPS.

There definitely will be games which are just unplayable if you got no inference accelerator. The moment, AI generates the actual visuals, you either are able to do AI image gen at playable FPS - or you don't. Those games can't ship with an alternative renderer because the game needing no textures or shaders is the whole point of full AI rendering.
But if you don't like AI, you will not miss out on anything. The first such games will probably be absolute dog shit anyway.