It depends on the device.

If the device requires an account and uploads data to the cloud, it's almost certain that they are recording your location (plus likely a heap of other sensor data). GPS receivers are in a lot of devices that are designed to track your location (like like smart watches, fitness trackers, cars, handheld GPS, PLBs, and so on). The tracking is often a key feature we want for our own benefit, we just don't want or need that data to be used against us by advertisers and governments.

If you have a device that only receives GPS and has no wifi, bluetooth, or other radio or network connection, and doesn't use a cloud account, then perhaps that device doesn't track your location. I'd bet that there aren't that many of those on the market though.