Yeah, you gotta keep dead limbs under control. Even mild winds can knock one down, and there's a limit to how accurate you can predict where it'll end up, as the giant dent in my car that was supposed to be far enough away can attest.
It always sucks to lose a tree, but a good tree service can usually keep one in shape enough to prevent it. If you lack access to that, just cutting it down entirely is sometimes the only viable option.
Another anecdotal point, my friend had a tree fall from their neighbour's yard. Took out an entire room, plus some structural damage, and his car. A year and a half later they were able to move back in. All the fuckery of dotting Is and crossing Ts was a few months of that. The neighbour's insurance covered all of it, but it was a pain in the ass.
Don't let trees slide if you have any choice at all.