post
What's "the privilege of running a local model"? If it's on a computer, it's not much of a privilege, gpt4all can get up and running in a minute. For mobile phones RAM is more of an issue.
Paid commercial LLM providers already profit off of inputs more than outputs, so I wouldn't trust any free cloud offering to be private. If you really want free and private, self-hosted is the only way to go
OpenRouter has some decently powerful free-to-use models, but I'm afraid as far as LLMs go 'free', 'good', and 'private' are going to be pretty mutually exclusive if you can't run one locally.
I use Jan. It's really good. A minimal install needs around 10GB of space and you can use it offline. The 'jan nano' model works faster than I can read in most cases.
There are a few AI models that are in this category, and I really liked Jan when I tested it out.
The Qwen models that you can run with llama.cpp (Jan's main backend) are quite brilliant for their size
http://duck.ai/ maybe?
"If something is free, you are the product"
If you're not providing your own compute power or your money (and you're not safe even if you pay most providers), expect the payment to come from your personal information or conversation contents being harvested. I'm not sure what magical service you're expecting that doesn't involve self-hosting for truly private conversations - you're just accessing someone else's machine if you're not hosting it yourself, and you're at their mercy.
nano-gpt.com
Venice.ai is an uncensored model which is free and needs no account. Not sure if it's truly private though.
top 19 comments