30
Kubernetes storage backends (lemmy.world)
submitted 1 year ago by to c/selfhosted@lemmy.world
23 comments fedilink hide all child comments
 

I'm looking for experiences and opinions on kubernetes storage.

I want to create a highly available homelab that spans 3 locations where the pods have a preferred locations but can move if necessary.

I've looked at linstore or seaweedfs/garage with juicefs but I'm not sure how well the performance of those options is across the internet and how well they last in long term operation. Is anyone else hosting k3s across the internet in their homelab?

Edit: fixed wording

you are viewing a single comment's thread
view the rest of the comments
[–] 9 points 1 year ago (3 children)

Longhorn is pretty easy to use. Garage works well too. Ceph is harder to use but provides both block and object storage (s3).

  • permalink
  • fedilink
  • source
  • hideshow 3 child comments
    •
  • [–] [S] 1 point 1 year ago (2 children)

    Ceph (and longhorn) want "10 Gbps network bandwidth between nodes" while I'll have around 1gbit between nodes, or even lower.

    What's your experience with Garage?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 1 year ago (1 child)

    I guess the network will be a bottleneck on Garage too. If you want high performance you might need a hybrid solution, like clustering of stateful apps on local storage as well as periodic full backups on a distributed storage.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] [S] 2 points 1 year ago

    It's fine if the bottleneck is upload/download speed, there's no easy way around that.
    The other problems like high latency or using more bandwith than is required are more my fear. Maybe local read cache or stuff like that can be a solution too but that's why I'm asking for what is in use and what works vs what is better reserved for dedicated networks.

  • permalink
  • fedilink
  • source
  • parent