Had to just think about why #[non_exhaustive] exists, so perhaps this helps you as well:

Imagine you're developing a really important core library, something like TCP. And then other libraries depend on your library.
Now, if you change your implementation, you may need to introduce a new error case, even though the rest of the API still works the same.

You have two choices to deal with that:

• Release version 2.0, because that's a breaking change. Each library that depends on you needs to manually upgrade to version 2.0. Some never will, because they're unmaintained. Any application that ships your library as a transitive dependency will need to ship both, version 1.0 and version 2.0, for the next few years until the last of their direct dependencies has upgraded to 2.0 or been removed.
• Or: You've declared that error enum #[non_exhaustive] before going 1.0 and therefore forced all depending libraries to implement a default match case. You can sneak in the new error case in version 1.1 without all those depending libraries needing to manually upgrade. The ecosystem does not split in half between 1.0 vs. 2.0.

Obviously, if it's a really egregious new error case that needs specific handling, then you might still want to release version 2.0, but by adding the #[non_exhaustive], you can decide later on and if you are such a central library, there's definitely going to be situations where you want to make use of it.

Maybe a default branch in the handler could do, though

This is exactly what it's for. #[non_exhaustive] forces you to add a default branch when matching on the error. It's already used a lot, so without a way to do that, it becomes less useful in library code.