Damn how old is your controller? My series X controller worked from day one. Zero configuration needed. Nobara Linux. I use it almost everyday for hours, zero problems

This is mostly a myth with modern controllers. Either your pc Bluetooth is very old, or you are using very old controllers. Modern bluetooth controllers connected to modern bluetooth receivers do not give any noticeable latency.

Well I use one Series X original controller that came with my Xbox series X on my Nobara desktop almost daily. Zero issues. Maybe it depends on the distro or kernel version. I didn’t have to install any drivers whatsoever

If you have bluetooth on your PC you can use any Xbox controller (from Xbox One up). They are the perfect standard. Grab some rechargeable double As and the controller will last literally forever and is compatible with everything. Or a battery pack even from third party brands. I know many people complain about it using double As but its actually a great thing, double As last 20x longer than any battery and you can use rechargeable ones and don’t worry about your controller dying as it ages.