As you mention, use a small fan filter to draw away the fumes. If you can smell the flux vapors, you don't have enough ventilation (although most people won't care). Unless you do long sessions I rarely see people wearing masks or gloves but if you are confortable doing so then yes that's a good idea.

One thing is glasses. Especially when inexperienced, it can happen that you can fling around small solder blobs by accident. You don't want that in your eyes.

I would suggest not avoiding flux. It's basically easy mode compared to without. Even if you use flux core solder.