Depends. Pure LLM, sure, you are right. LLMs are a terrible way to "store" information.

Coupling LLMs with a decent data source on the other hand isn't such a terrible idea. E.g. answer the question with a google search summarized by LLM can work.

The bigger issue here is (a) when it doesn't seach but does everything locally and (b) that now the site owners lose traffic without compensation.