Well, yes, they should use open weight models.
…But on prem?
Honestly, finetuning modern MoEs to actually improve at tasks (much less without overfitting) is very difficult. It’s why finetunes are so rare in HF now, even though we still some for Qwen 27B.
Even deploying such large models is hard.
That, and inference costs decrease at scale (to a certain point). It’s way cheaper for some inference company to keep dozens of servers saturated, with parallel requests split across many clients, than for a company to leave a bunch of mostly-idle servers barely spun up.
In privacy insensitive scenarios, open weight APIs make sense for most companies (IMO), and the great thing about that is you can pick from hundreds of providers. Rates are highly competitive. You aren’t locked to a single source like the “frontier” models.