What video card is it? (Vendor too)
What processes are running?
What CPU do you have?
Have you modified any fan curve settings or overclocked? Do you have any software that may do that behind the scenes (x1 or afterburner for example). Have you messed with the card's heatsink?
Does it need dusting? How's the airflow? Is the computer in a good spot?
Are there any vbios updates from the vendor? Drivers up to date?
That's all the basic considerations I can think of. I also know that it's normal for windows to utilize 3d settings for transparency on windows gui and other random background tasks. Not to the extent it would sound like a vaccuum cleaner, but that may be able to be addressed by a software fan curve. My evga 3090 is horrible at managing its own fan temps and I had to set a much more aggressive curve to avoid lost frames and obnoxious fast then slow fan speed that destroyed 2/3 of my fans before I caught it cycling (I run folding in the winter so mine is rarely idle)
Edit: After reading your edit, I'm pretty convinced it's a fan bearing. Those slip bearings kinda suck at dealing with heat from a modern system that runs hotter (ryzen especially). Check your psu fan too, literally every fan in the system. Only hdds and fans (I guess water pumps too kinda) make that noise.
A big gotcha with fan bearings is that it's usually just resonance. If the fan speed changes the noise may go away. You need to have the system open and ready to check the next time you hear the noise. You can also step up the speed with a fan curve app.