Loved the replies. Once it’s over (that’s the hard part), everyone has a funny story to tell!
Mine was a race condition in an outsourced core system that showed up about six months after release, only when enough transactions hit at the same moment. The contact centre was on fire, with people calling non-stop.
Same lesson as yours: when something fails only sometimes, stop staring at the code and look at what else is running at that moment.