I think it is good to have optional support for local models that lets people use them in an offline and private and easy way. There is a lot of non technical folks using linux nowadays and many chose it for privacy and greater control over their data.

Depending on the implementation it could hook into certain os contexts and events to actually be helpful.

Either way I don't see the cat going back in the bag with regards to LLMs. That being said I run Debian everywhere except my work machine which is ubuntu.