I'm a bit of a nerd who has had a bit of experience in PCs and Linux, so my recommendations would be pretty low-level, but if that's what you're into, then it's worth sharing.
I use an old inherited mid-2012 Dell desktop PC with a few modifications, using 100% all old used hardware. A 10gb network card (not necessary, the default 1gb port is likely enough), an old 2015-ish NVIDIA workstation card ($50), a used SSD for the Linux OS (Ubuntu server), and 2 used HDD for media storage. Roughly $100-200 in hardware. Installation of the OS can be done by flashing a USB drive, or you can learn how to use network booting to get the job done if you don't have any flash drives. After the initial setup of Ubuntu Server, I install OpenSSH on it and do the rest of the work via SSH from my phone (AnotherTerm on Android has been my favorite) and never touch the PC/server manually again.
Install Docker, and follow the Jellyfin docs on installing the proper NVIDIA drivers, FFMPEG, and so-on from their website.
From there, once Docker is working, I prefer to install "Dockge" so I can manage my Docker "stacks" very easily in the form of a locally hosted website that allows for easy management of the Docker service YAML files, updates, and taking the services up and back down.
After that, look into the "Trashy Guides", those guides are essential to getting the whole thing put together and tailored to your own build.
Good luck and have fun, it was an incredibly rewarding experience and still something I get excited about.