There are so many games on Steam and every day a few hundred more are added. I assume there are automated checks and rudimentary malware scans in place but those aren't fault proof.

This appears to have originally been published as a totally different non-malware game. Either the original dev got their account taken over or turned heel, because the entire game was replaced with the malware game as an update to an existing game rather than a new published game.

I'm only speculating as I don't know much about the Steam publishing process, but I wonder if that helped the malware sneak past more rigorous checks which would happen on a totally-new upload.

When I first published a game on steam, valve kept blocking it because I had checked "controller support" and they tested it and said it didn't work with controllers. I tried to find any controller that didn't work, asked a lot of people to test it for me as well, no issues whatsoever. Gave up and unchecked that option. Game got approved. Players used controllers just fine, I went back and checked it and never heard anything from valve again.

Scanners are only going to pick up known "off the shelf" malware. They are never going to pick up something bespoke that the developers wrote themselves.

Couldn't they just put the malware in encrypted compression files that the game unpacks on the client end?

With the amount of games published every day, they can't. They should, but really can't. Either they keep it this way, or review each and every game under the Sun to find malware before they get published.

Maybe? Games are huge nowadays and looking through all of them will probably be impossible and not sure how well it'll prove? Google does that and there still are a lot of malware on play store.