From the sbcgaming subreddit, the one I see constantly popping up is the TrimUI Brick (Hammer for metal case).
Of course any retro games handheld with Android will work just as well as any other phones with Android. And you can just put it on airplane mode or just don't install any apps that'll give notifications.
Additionally, there's a rockbox app for portmaster.