Come back after you've gotten the pogo stick. Then you end up jumping through the window above and while you're happy to have unlocked new ground to explore, you're a bit disappointed that you can't yet consider that door solved.

Then later on you don't even notice that the a boss flings you through that door from the other side and get confused whrn you return even later on and see it just smashed and you don't even recognize the room beyond without the giant boss on it.