French for me. Although I remember learning "pan d'or" was more like a French toast, but I dunno. So when I saw pandoro for the first time this Monday I'm ShopRite, it clicked for me the same way it did for you.

That being said, I'm from Jersey and was raised in traditional Italian American fashion, and neither pannetone nor pandoro were ever part of our family's traditions. My grandma focused more on mountains of dinner; dessert was just an afterthought.