What is everyone using for the LLM model for HA voice when selfhosting ollama? I've tried llama and qwen with varying degrees of understanding my commands. I'm currently on llama as it appears a little better. I just wanted to see if anyone found a better model.

Edit: as pointed out, this is more of a speech to text issue than llm model. I'm looking into the alternatives to whisper

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 1 year ago (6 children)

The Gemma 27b model has been solid for me. Using chatterbox for TTS as well

  • source
  • hideshow 6 child comments
  • [–] 2 points 1 year ago (4 children)

    27b - how much VRAM does it use?

  • source
  • parent
  • hideshow 4 child comments
  • [–] 2 points 1 year ago (3 children)
  • [–] 2 points 1 year ago (2 children)

    So basically for people who have graphic cards with 24GB VRAM (or more). While I do, it's probably something most people don't ;)

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 1 year ago (1 child)

    Yeah, I went a little crazy with it and built out a server just for AI/ML stuff 😬

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 year ago

    I could probably run something on my gaming PC with 3090, but that would be a big cost. Instead I've just put my old 2070 in an existing server and using it for more lightweight stuff (TTS, obico, frigate, ollama with some small model).

  • source
  • parent