Definitely be careful though! There's plenty of rail that's overgrown, horribly maintained, looks like it hasn't been touched in 20 years but gets a train every few weeks or something. You'll want to be very sure that it's impossible for a train to be on the same track.

(There's a town in my state that has some rail that is physically cut off from the railroad it used to be part of, so something like that would be the safest)