Ok, yeah, I mixed up Foveated Rendering and Streaming.

So what I'm trying to say is that Valve has figured out a generalized Foveated Streaming layer that just works on all games, full stop.

So, you, a game dev... look at that, and build around whatever they're using in Steam Link to accomplish that... use that as a template/basis to build a sort of dual device , seperated out async threading mode.

If Steam Link can move data that fast, then build your game/engine from the ground up with that protocol in mind, that you're always gonna have a second hardware instance that you can run some subportion of game logic / engine calls on, some kind of async multithreading.

Not just native Foveated Rendering in a VR context, but potentially any kind of game/engine level sort of API or... mode or something, that would allow this superfast Steam Link streaming to have a game be ... collaboratively generated by many devices linked with this kind of nearby, wireless speed.

Maybe a rough analog here would be the difference between a game that is built from the ground up with full Steam Input support, so it gives you more options than if you're just using Steam as an over the top, generalized input translation layer.

Of course, ... I think basically only Source games actually have full Steam Input support, and I have no idea if this Foveated Streaming thing is... open source or proprietary to Valve.

If its open, theoretically anyone could try to do this.

If its proprietary, well then it'd just be Valve.

Maybe another analogy that comes to mind is a weird, little used way of running ARMA servers.

So with ARMA games... they're big, fuckass complex, open world milsims, ARMA literally just is the commericial, streamlined version of the much more hardcore milsims they sell to governments/militaries.

ARMA servers can be ran in a way where you have more than one server doing the actual computations. This mode, these protocols, they do exist in the commerical variants, but they are not well documented, almost nobody uses it, because its quite complex.

Players will join the main dedicated server, but the dedicated server can also shunt off tasks like simulating the movement patterns of 100s or 1000s of other NPCs to other, synchronized servers.

So what you as the player join as the dedi, thats basically just coordinating traffic and world states between players, and also NPCs, who are essentially being simulated on their own servers, which are running in a mode that is optimized for just simulating them.

This is kinda sorta analagous to what I'm trying to describe with the Frame and Machine, its just thst now you as a player, externally networked or not, now you have a two hardware system combo that is cumulatively producing your whole game experience, which theoretically should be possible if you can low level implement whatever protocol Steam Link is doing to be able to stream data both ways so fast, as they are with Foviated Streaming.

Right?

Because that Foveated Streaming has to be hooking into actual hardware inputs from the Frame, the eyetracking, then it bounces that to the Steam Machine, does calcs, throws some result back to the Frame, super fast.

So, it has some low level of hardware input access, and obviously the Frame alters what the display renders based on what the Machine tells it to, so we have two way, super low latency, local wireless communication going on here.

From the Gamers Nexus interview, basically, this is only possible because the Frame and Machine are talking via a dedicated WiFi7 dongle, that is soley used by them.

Anyway, no clue if this would actually make in practical sense in terms of trying to take the time to develop it, what the actual gains or benefits would be... I'm just trying to brainstorm out things that may potentially be neat/possible.

Well, like the examples I tried to give... its fairly common in game dev to set aside a distinct physics thread or set of threads, that run at some fixed number, maybe 30 or 60 fps, to avoid the once very common problem of game physics functions going bonkers with unexpected hardware speeds.

It also helps with just generally normalizing physics behavior in complex vs simple scenes, lets you just have solid bounds for number of whatever kind of dynamic things possible in an area.

Or, for doing things like deep simulation, where you run like, an aporixmation, a simplification of many npcs, non rendered, but when a player comes into view of one of them, then they transition from approximately simulated, to fully simulated.

So like, most games just spawn and despawn like crowds of people by more or less a 'where are you' 'what time of day is it' and 'what npcs should be here', and then it just picks blank templates out of a pool.

But if you wanted a uh, 1000s of npcs doing a daily routine in a city or landscape, potentially having reactions to other npcs they may meet... thats more like deep sim, thats something you can approximate on a slowed down timeframe, thus potentially shunt off to another segment of threads/hardware, more geared toward regular iterations on a massive dataset, than hyperspecific, hyper fresh data.

Can also be used with something like dynamic, ambient loot or wildlife or fauna generation/simulation.

Like yeah, I agree that there are mant uses cases where 10ms is still too much latency, but... why not apply 'foviated rendering' to something like... procedural terrain generation, in chunks/maps/meshes of descreasing size, increasing detail?

If the player is mostly stationary, not moving rapidly, just slowly build out more detail ariund them, in a circle.

If they are moving rapidly, prioritize pre caching various detail levels of chunks that they are rapidly moving toward, stop the above naive, layered detail build out process.