you are viewing a single comment's thread
view the rest of the comments
[–] 7 points 5 months ago

I think it's a bit early to be locking down your local LLM setup. The history of computing suggests there is some cost reduction to come for the hardware. In the mean time there is time to find the sweet spot for performance of the small and medium models. There are plenty of cloud hosts which can run the open models and allow you to experiment while the models mature. Hopefully your not burning 6k worth of tokens anytime soon.

  • source