Even if it is the "best" it gives one target to try to find a weakness in, rather than many.
It sounds as if you're falling prey to the allure of security through obscurity.
I'd like to remind you of Kerckhoff's Principle.
a cryptosystem should be secure, even if everything about the system, except the key, is public knowledge
What this is saying is that in a secure cryptosystem all you need to secure is the key.
Curve 25519 is, for the time being, considered a secure cryptosystem. It's likely that in the future it will become obsolete, but we're not there yet. When we get there WG will port to a new cipher that is more secure.
Even if you set your configs to not negotiate, just the fact that the stack COULD negotiate opens you up to downgrade attacks. Also, anybody trying to connect would get some kind of hello/response traffic that would broadcast your (non negotiable) configs, so what do you gain from being able to choose, except the ability to make mistakes?