Ah yes, I agree that's a fair way to look at it. I think if you combine that reasoning with the idea of a deterministic universe, then you also basically arrive at Spinoza's view where it is said that God is Nature, because effectively Nature then is the constraining agent.

But it does still lead me to question the sentence you posited at the start:

Whether that’s compatible with a deterministic universe then depends on whether there’s an omniscient agent in your environment.

If such a constraining agent does not exist, doesn't that imply that the universe is not deterministic? Unless you also believe that free will can exist in our mind without the ability to act upon the universe. This too was argued by Spinoza, though personally I never found such an argument to be convincing.