you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 6 months ago

Or you mean local HTTP?

Yes, I mean local HTTP. ollama listens on port 11434 and responds to HTTP requests by default. I'm not sure what llama-server uses by default, but like I said, I'm pretty sure you can do the same (or at least something very similar) with it.

I was actually asking more model type.

OK, I see what you mean. I'm still too new to LLMs myself to have a good answer then on that beyond saying that I know it works with qwen3.6 and gemma4 from having actually experimented with those specifically.

I did a little nlp

I mean, something like:

result = result.replace("```json", "").replace("```","")

is good enough in practice for the kinds of things I've been doing. (I'm dealing with cases where triple backticks should never appear in the output though; you might have to get more creative if you want a result that has that kind of quoting embedded in something else...)

  • source
  • parent