I think you're over estimating the amount of damage a fresh grenade would do. You're likely only hitting the outside of the house. That's probably just some siding and some window damage.