No need for the Pico-sized Linux board to have Pi5 performance. It's fine if its performance is like a single core version of the Pi Zero 2, with say 64MB of memory instead of 512MB. Just get the size, power consumption, and cost down.
Reasonable points about the CM boards but I don't see much attraction of the CM0 vs CM4 or CM5. Given that they aren't after cost minimization, might as well take the extraperformance.