Selfhosted

63,723 readers

5,179 active

712 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

Be civil.
No spam.
Posts are to be related to self-hosting.
Don't duplicate the full text of your blog or readme if you're providing a link.
Submission headline should match the article title.
No trolling.
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

moderation log

moderators

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world

Selfhost an LLM (downonthestreet.eu)

submitted 1 year ago by Shimitar@downonthestreet.eu to c/selfhosted@lemmy.world

20 comments fedilink hide all child comments

Hi all, i am quite an old fart, so i just recently got excited about self hosting an AI, some LLM...

What i want to do is:

chat with it
eventually integrate it into other services, where needed

I read about OLLAMA, but it's all unclear to me.

Where do i start, preferably with containers (but "bare metal") is also fine?

(i already have a linux server rig with all the good stuff on it, from immich to forjeio to the arrs and more, reverse proxy, Wireguard and the works, i am looking for input on AI/LLM, what to self host and such, not general selfhosting hints)

you are viewing a single comment's thread
view the rest of the comments

[–] muntedcrocodile@hilariouschaos.com 2 points 1 year ago* (last edited 1 year ago)

Openwebui is awesome and allows u to use it as an api for all the models u have it hooked up to. Can point it at ollama or any openai api compatible endpoint (like open routers)

permalink

fedilink

source