Dragon's Lair was a hugely popular arcade game that worked that way.

https://en.wikipedia.org/wiki/Dragon%27s_Lair

I thought that they were managing that stuff on a per-pixel basis, no engine, assets, or other abstractions, just raw-dogging pixel colors.

And before I even played video games at all I was watching somebody play some assassin's creed game I think and I thought the player had to control every single limb qwop-style.

Even with 2D games that's basically impossible. Only time it could work is with turn based games and then...you end up with this post lol.

I remember having a thought one day as a young kid while interacting with a DVD main menu (the kind that had clips from the movie playing in the background, and would play a specific clip depending on what menu you went in to).

"This is basically how video games work, there's a bunch of options you can choose from and depending on what you do it shows you something. Videogames are just DVD menus with way more options."

I grew up to not be a programmer.

Its not terribly far off from pre rendered or FMV games like Myst are doing it.

Those shitty old tiger hand holds kind of worked like that.

It does work like that a little bit, like with sprites they've often hard-coded the frames of animation, so when you push a button it loads the correct image, like Mario's jumping frame with his hand in the air. But there are such things as tilesets, and sprite positions, and all that good stuff.

I remember speculating as a (small) kid that the AI soldiers in Battlefront II's local multiplayer might be real people employed by the developer. Not the brightest child was I.

This is what I believed. And I tried to trick the game by doing movements and inputs no one could have planed. Never outdid that planning somehow. They were on to me!

I grew up mostly with the PS2 and above and I thought the same thing 😅. I did think there had to be a better way though