When I was a kid I used to make bows and arrows, and so obviously lit arrows on fire.

You for sure need an accelerant. I recall personally using gas line antifreeze. Otherwise yeah they'll extinguish pretty much instantly.

I can't really remember if the flaming arrows "flamed" the whole flight, or if they extinguished in flight but because of the accelerant were able to reignite once they stopped.