No, you don't have to pay. There is a free self-hosted option using docker.

Joplin Server is basically a postgres database sync with a user management frontend.

You don't need to use it with a reverse proxy. For the first two years I just used it on my local network's ip. If I was away from home, I vpn'd and synced that way. But now I use it with a reverse proxy and everything is much simpler that way.