Gemma 4 12B QAT + faster Qwen 3 TTS fits perfectly fine in 12GB VRAM and can be used to create a fully local voice assistant with ... reconizeable traits ;) The project is fully open source and published here: https://git.sync.wtf/starfleet I see Youtubers trying to do similar things every now and then but for some reason their latencies always seem a lot higher than I would expect so if the one I've conjured up (based on Huggingface's Speech-to-Speech project initially) is special somehow then maybe it's of interest to others.

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 2 months ago

Ooh seems great !

I am nowhere near my compter (hah !) right now so I can't test, but last time I checked, qwen3 TTS was NVIDIA (well CUDA) only

Will my full AMD rig on linux be able to run it ?

  • source