916
The kids are going crazy for these things (lemmy.world)
submitted 4 months ago by to c/memes@lemmy.world
53 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 4 months ago (4 children)

A Proxmox cluster with Ceph pretty much won't work without 10Gbps.

It'll slowly fail and all your data will get lost. Ask me how I know.

(Also, you need at least 4GB memory dedicated to Ceph on each node... Another compounding lesson learned)

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 4 points 4 months ago (3 children)

    If software depends on specific infrastructure to work, then it should have it as a hard requirement. Haven't tried Ceph myself, but this sounds like there's something very wrong with it conceptually or the way it's set up.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 3 points 4 months ago (2 children)

    It's more that Proxmox has a weird niche laying between both enterprise/small-medium business and also homelab infrastructure.

    Ceph is designed towards the former where 10Gbe fibre and hosts with a boatload of RAM is fairly commonplace.

    My problems were a result of not paying close enough attention to those requirements and just hoping it would work on small micro PCs with 12GB of ram.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 4 months ago (1 child)

    Yeah, but even if that's the case, I would expect it 1) to refuse to run at all if something's not fit or 2) to just be bottlenecked by the network instead of making data disappear altogether.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 4 months ago

    Oh, to be fair the data disappeared because I was stupid for a different reason and proxmox upgrades were done out of order.

    I'm basically 100% to blame for the data loss, but it was impacting performance from an IO and memory perspective before I fucked it all up.

  • permalink
  • fedilink
  • source
  • parent