kde can be controlled with a gamepad (tho it’s a setting you need to activate first), but in my experience it was pretty janky. maybe it’s better now? it’s an option to consider, at least
they also have plasma bigscreen which works better with a controller, but since it’s pretty recent (iirc it only got added in plasma 6.7, the latest major release) i doubt it will be in debian’s repos