NaN is one of the ways IEEE maths signals an erroneous calculation when the only outbound communication channel is the returned IEEE value. Reading the history and rationale of the floating-point standard will likely convince you this was and probably is a good compromise.

Because IEEE values often have hardware acceleration that is a significant factor in performance, languages rarely (if ever) choose to map all IEEE error values to language-specific error reporting (e.g. a thrown exception or error-tracking algebraic data type).

JS is hardly an outlier here.

I do think it probably violates the principle of least surprise, but I don't think I understand the scale of "momentum" change that would need to take place across society (industry, academia, etc.) to align it with naive expectations.

NaN is a error and it’s your responsibility to handle it gracefully. No need to halt the entire program when you fuck up some maths.

No, it should not be an error. If you're making a calculator and your user enters 1/0, your program should not have the propensity to crash; the result should simply be NaN and you should choose how you display that result to your user.