Your brain is weirdly unhappy with the concept of other brains doing math. Nice!

The brainpower involved in mental math is just real-time factorization and otherwise general application of the rules of arithmatic. It's no waste, it's just a well practiced set of pathways that take annoying math and make it friendly. It takes at most 2 seconds to know if it's time to use a calculator.