You can do it locally now pretty easily depending on your use ass and hardware, huggingface has all the models you'd need and use something like llama-swap
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments