you are viewing a single comment's thread
view the rest of the comments
[–] 12 points 3 months ago

Open models are still most cheaply hosted on a cloud, with batching and 24/7 use. API rates from developer lab are generally fair. Self hosting does have some significant tangible benefits though: Fine tuning for domain specific to organization, and not letting LLM provider train from your prompts/answers, followed by competing with your organization in the future as a result of "distilling your IP".

  • source