Ok, so bear with me, I have this orchestration tool i call Noesis that's built in Rust. It does a ton of stuff, but what it's basically for is figuring out where to place AI workloads. It knows... "I have four linux hosts, here's their system RAM, here's their CPU, here's what GPUs they have." If a client that wants a voice is on the network, it asks Noesis to place chatterbox on a GPU and Noesis does so. Then it tells the API (which serves prompts, memories and endpoints) "Ok, there's a websocket on at where chatterbox is available." That websocket is a Noesis instance that then yes, passes incoming text to the chatterbox CLI and returns an audio stream. Anything on the network with the Noesis fleet key can hit that websocket and ask for audio.

Noesis has some ability to prioritize stuff (it knows scientific models are more important than tts which is more important than lipsync animations), but my local model can also tell Noesis she wants a voice and kick something out. For example, when my son comes in the room and I'm not there, she gets a prompt telling her he came in and she should say hi and ask him what he needs, and she can (and has) kicked out other workloads to get a voice to use for that. She can also turn her animations on and off and wake the monitor, so yes, when my teenager comes in the room the monitor wakes up and there's a very uncanny valley AI scientist girl on it who lipsyncs and says "Hey, what's going on? Did you need something? Want me to call your dad?" Which is dope, but like, the very bottom of the stack of cool things she can do.