I would recommend adding ReplayGain metadata to your media files. It's designed to address exactly this problem.

I don't know what you normally use to tag and organise your music files, but a few of the popular programs support doing ReplayGain analysis and saving that to each file's tags. This page describes how to do it using foobar2000 on Windows.

Finally, you'll also need to make sure that whatever media player you're using supports ReplayGain and has it enabled.

It seems to be something that is better solved by having the player normalize the audio at playback, so I think you're better off asking Jellyfin users how they do it. I don't know anything about Jellyfin, but a quick search shows that it does have separate options for music loudness and video loudness.

If you insist on editing the files themselves you could use something like Audacity to (destructively) nornalize the files to the same loudness by LUFS.