Not knowing the name of a bug is barely evidence of anything, but otherwise I agree. I've never seen any kind of wasp nest or even dried mud period that was actually brown and not a whitish, ashy, tan to gray.

I wanna see video of the nest attempting to be crushed to prove it's concrete. Actually scratch that. It doesn't even look like it was made by an insect; it could be concrete but was made by human hands for the bit. The holes are too perfect.