20
What are the best options for local conversational voice agents? (programming.dev)
submitted 1 year ago by to c/localllama@sh.itjust.works
8 comments fedilink hide all child comments
 

I found this project which is just a couple of small python scripts glueing various tools together: https://github.com/vndee/local-talking-llm

It's pretty basic, but couldn't find anything more polished. I did a little "vibe coding" to use a faster Chatterbox fork, stream the output back so I don't have to wait for the entire LLM to finish before it starts "talking," start recording on voice detection instead of the enter key, and allow interruption of the agent. But, like most vibe-coded stuff, it's buggy. Was curious if there was something better that already exists before I commit to actually fixing the problems and pushing a fork.

all 8 comments

sorted by: hot top controversial new old
[–] 2 points 1 year ago (2 children)

I got a bonus question... Is there a good end-to-end voice conversation solution? I'd like to try something which directly processes the audio and returns audio, rather than the whole pipeline with vad -> stt -> llm -> tts

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] 4 points 1 year ago (1 child)

    There are not many models that support any-to-any, currently the best seems to be Qwen3-Omni, the audio quality is not great and it is not supported by llama.cpp: https://github.com/ggml-org/llama.cpp/issues/16186

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 1 year ago

    Thanks! if anyone has more (good) alternatives or something like a curated list, I'd have a look at that as well... always a bit complicated to stay up to date and go through the myriad of options myself...

  • permalink
  • fedilink
  • source
  • parent
    • [–] 3 points 1 year ago (1 child)

    I haven't followed that closely lately, but KoljaB on github had some interesting repos. Also 'livekit' is new and afaik creates an audio agent. No personal experience though..

  • permalink
  • fedilink
  • source
  • hideshow 1 child comment
    •
  • [–] 1 point 1 year ago*

    Nice, that's quite an assortment of random stuff from speaker diarization to full-blown virtual assistants with avatars. Probably worth having a look when tinkering around with Python and AI. By the way, Livekit is a WebRTC video conferencing toolkit. I didn't know they had AI agents in there. Guess I can build my own callcenter now. Or the AI granny (Youtube), I've always wanted something like that answer my landline calls. 😆

  • permalink
  • fedilink
  • source
  • parent
    • [–] 2 points 1 year ago

    Kobold.CPP has pretty good TTS model integration I used OuteTTS model when I played around with it but theres also API integration with commercial ones like kokoro.

    However, I'm no sure if its able to stream to a TTS model as the llm is generating when I tried it just waited till after output to send to voice model you may need to do some documentation reading to see if real time streaming is possible if you go that route.

  • permalink
  • fedilink
  • source
    •
    [–] 1 point 1 year ago

    Alpaca on Flathub seems ok

  • permalink
  • fedilink
  • source
    •
    [–] 1 point 1 year ago

    OpenWebUI has TTS and STT.

  • permalink
  • fedilink
  • source
    •