86
Replaced $40/month in AI API subscriptions with self-hosted Ollama + n8n (discuss.tchncs.de)
submitted 7 months ago by to c/selfhosted@lemmy.world
38 comments fedilink hide all child comments
 

Quick post about a change I made that's worked out well.

I was using OpenAI API for automations in n8n — email summaries, content drafts, that kind of thing. Was spending ~$40/month.

Switched everything to Ollama running locally. The migration was pretty straightforward since n8n just hits an HTTP endpoint. Changed the URL from api.openai.com to localhost:11434 and updated the request format.

For most tasks (summarization, classification, drafting) the local models are good enough. Complex reasoning is worse but I don't need that for automation workflows.

Hardware: i7 with 16GB RAM, running Llama 3 8B. Plenty fast for async tasks.

you are viewing a single comment's thread
view the rest of the comments
[–] 22 points 7 months ago (9 children)

No, not free, OPs power bill just climbed behind the scenes to match. Probably a discount but definitely not free.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 9 child comments
    •
  • [+] 20 points 7 months ago (5 children)
    [deleted]
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] -3 points 7 months ago (4 children)

    Running a thousand watts and not running a thousand watts can be quiet a difference depending on where you live. And then consider buying all of the hardware. In many cases it's probably cheaper to just pay $40 al month.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 9 points 7 months ago*

    Do you think it runs at 1000w continuously? On any decent GPU, the responses are nearly instantaneous to maybe a few seconds of runtime at maybe max GPU consumption.

    Compare that to playing a few hours of cyberpunk 2077 with raytracing and maxed out settings at 4k.

    Don't get me wrong, there's a lot to hate about AI/LLMs, but running one locally without data harvesting engines is pretty minimal. The creation of the larger models is where the consumption primarily comes in, and then the data centers that run them are servicing millions of inquiries a minute making the concentration of consumption at a single point significantly higher (plus they retrain the model there on current and user-fed data, including prompts, whereas your computer hosting ollama would not.)

  • permalink
  • fedilink
  • source
  • parent
  • [–] 3 points 7 months ago (1 child)

    While this is correct, sometimes it can be free. I live in a cold climate, and over the winter I hooked up a folding@home computer in my office to keep things a bit warmer. Computers are 100% as efficient as a space heater.

    Of course now that it's getting warm things are changing. I'm actually in the middle of doing my last folding@home tasks until the temps drop next fall.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 3 points 7 months ago

    Even in very cold regions heat pumps maintain a COP>1, so even running it as a space heater may not be free if you have access to a more efficient alternative. Also that may be a responsible justification for Folding@Home, but I doubt OP is turning off their LLM in the summer.

  • permalink
  • fedilink
  • source
  • parent