sorted by: new top controversial old
-48
Why We Chose Self-Hosting Over Cloud APIs (lemmy.world)
 

While building PolyTalk, one of the biggest decisions we faced was whether to rely on cloud APIs or keep everything self-hosted.

At first, cloud services seemed like the obvious choice. They make it easy to get started and remove a lot of operational overhead.

But the deeper we got into the project, the more we realized that self-hosting wasn't just a deployment preference, it was a requirement for many of the use cases we were exploring.

A few things we learned along the way:

  • Running speech recognition, translation, and TTS locally is absolutely possible, but latency quickly becomes one of the biggest engineering challenges.
  • Supporting multiple audio sources (microphones, meetings, browser tabs, system audio, etc.) is often more complicated than the translation itself.
  • Choosing models is a constant trade-off between quality, speed, hardware requirements, and language coverage.
  • Privacy, compliance, and data sovereignty concerns came up far more often than we expected when talking to potential users.

Self-hosting definitely isn't the easier path. You have to think about infrastructure, updates, monitoring, and resource management.

That said, the trade-off is greater control over your stack, fewer external dependencies, and more flexibility in how the system is deployed and operated.

For us, those benefits were worth the extra complexity.

I'm curious how others in the self-hosting community think about this.

When do you decide a service is important enough to self-host instead of relying on a managed API or SaaS provider?

For anyone interested, PolyTalk is the project that led us down this rabbit hole:

GitHub: https://github.com/PolyTalkIO/polytalk

Website: https://polytalk.io/-

What We Learned Building a Self-Hosted Speech Translation Platform in c/opensource@lemmy.ml
[–] [S] 1 point 4 months ago (1 child)

Thanks! You're right that there are already excellent open-source STT, translation, and TTS projects. PolyTalk isn't trying to replace them, and we build on top of them. What we're focused on is creating a complete, self-hosted real-time communication platform that ties those components together and can handle different audio sources (microphones, meetings, browser tabs, system audio, etc.) through a single workflow.

Regarding latency, we're not targeting sub-200 ms. In our testing, we've intentionally favored translation quality and conversational flow over minimizing latency at all costs. Depending on the setup, end-to-end latency is typically around 2 seconds. One thing we've already improved is processing complete sentences rather than translating word-by-word. That gives the translation more context and generally produces much more natural results. We're also working on additional context-aware translation improvements, and tone adaptation is on our roadmap.

  • permalink
  • fedilink
  • source
  • parent
  • context
    • What We Learned Building a Self-Hosted Speech Translation Platform in c/opensource@lemmy.ml
    [–] [S] 2 points 4 months ago

    Thanks! We're still benchmarking different setups, so I don't want to give a misleading "minimum spec" number yet. In practice, the hardware requirements depend much more on the STT/translation/TTS models you choose than on PolyTalk itself. For a single-user setup, you don't necessarily need expensive hardware. As you push for lower latency, larger models, or multiple simultaneous streams, the requirements increase pretty quickly. Proper hardware benchmarks are something we plan to publish once we've tested a wider range of configurations.

  • permalink
  • fedilink
  • source
  • parent
  • context
    • 21
    What We Learned Building a Self-Hosted Speech Translation Platform (lemmy.world)
     

    Over the past few months, we've been working on a project called PolyTalk.

    The original goal was pretty simple: make real-time multilingual communication possible without depending on external translation APIs or cloud-only services.

    While testing existing solutions, we noticed that many of them required sending conversations through third-party infrastructure. That works for some use cases, but it wasn't a great fit for organizations that care about privacy, deployment flexibility, or keeping communication workflows under their own control.

    So we started building a self-hosted, open-source speech-to-speech translation platform instead.

    A few things we've focused on:

    Real-time speech translation Self-hosted deployment Open-source core No external translation APIs Live audio translation

    The project is still evolving, but it's been interesting exploring the challenges of multilingual communication, local AI infrastructure, and real-time translation workflows.

    I'd be curious to hear how others here approach translation.

    Are you using cloud-based services, self-hosted tools, or something in between?

    GitHub: https://github.com/PolyTalkIO/polytalk

    Website: https://polytalk.io/