Yup, they can only dump heat through infrared radiation. The giant white fins on the ISS are radiators for waste heat.

"How do we cool these things?"

"idk, boil the oceans?"

Underwater datacenters make cooling very effective and maintenance nearly impossible, so you have to treat the container data centers essentially disposable. That's only viable with economy of scale big enough to be an xkcd comic punchline. I guess Microsoft found that even they are not quite there yet. Also most computers don't tolerate seawater quite as well as they tolerate air.

He touches on that in the blog post, and gives us this link: https://www.itpro.com/infrastructure/data-centres/microsoft-scrapped-its-project-natick-underwater-data-center-trial-heres-why-it-was-never-going-to-work

Didn't Meta, then Facebook, open a datacenter somewhere in the Arctic circle and leave it exposed to the environmental cold? Until the planet runs out of cold, an eventuality whose impending rapidity I'd rather not ponder, that seems a viable approach.