Has anyone tried in organization to use self hosted llm models for agentic programming?

Im curious if it makes any sense. My organization spends fortune on tokens from us companies. I want to recommend something…

you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 5 months ago*

it's an h100, I think, no idea about how many users

in my personal setup i use quantized versions on a 3080, which is not great, so I still lean a lot on APIs

  • source
  • parent