Yeah, there are, but I have only ever seen them on and between trees, never on houses. Would be nice to have them visit during the day, when my shutters are rolled up so I can admire them.
The hole is also very small and I had blackbirds dig in the pot next to it before, so I also suspect birds. Crows are known to leave gifts and acorns are among the possible choices.
If it was a squirrel, it may come back and search it in a few months.