I'm not sure why you'd NOT want a battery powered one, because as you say, the batteries last years, and a USB powered one would waste much more electricity over time, but I digress.
If you have any mini SoC boards or ESP32 devices laying around, you can get dirt cheap 2-pin external temp sensors. Works on anything with a GPIO pin out that supports simple I/O.
Anything you find that is Zigbee by nature will be battery powered, however, you might be able to find a device with a different purpose that happens to have temp sensors that export data you could monitor.