you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 9 months ago* (last edited 9 months ago) (3 children)

are you are saying running docker in a container setup(which at this point would be 2 layers deep) uses less resources than 10 single layer deep containers?

If those 10 single layer deep containers are Proxmox’s LXC containers then yes, absolutely. OCI containers are isolated processes that run single services, usually just a single binary. There’s no OS, no init system. They’re very lightweight with very little overhead. They’re “containerized services”. LXC containers on the other hand are very heavy “system containers” that have a full OS and user space, init system, file systems etc. They are one step removed from being full size VMs, short of the fact that they can share the hosts kernel and don’t need to virtualize. In short, your single LXC running docker and a bunch of containers inside of it is far more resource efficient than running a bunch of separate LXC containers.

One of the biggest advantages of using the hypervisor as a whole is the ability to isolate and run services as their own containers, without the need of actually entering the machine

I mean that’s exactly what docker containers do but more efficiently.

I can just snapshot the current setup and then rollback if it isn't good

I mean that’s sort of the entire idea behind docker containers as well. It can even be automated for zero downtime updates and deployments, as well as rollbacks.

When compared to 10 CT's that are finetuned to their specific app, you will have better performance running the CT's than a VM running everything

That is incorrect. Let’s break away from containers and VMs for a second and look deeper into what is happening under the hood here.

Option A (Docker + containers): One OS, One Init system, one full set of Linux libraries.

Option B (10 LXC containers): Ten operating systems, ten separate init systems, 10 separate sets of full Linux libraries.

Option A is far more lightweight, and becomes a more attractive option the more services you add.

And not only that, but as you found out, you don’t need to run a full VM for your docker host. You could just use an LXC. Though in that case I’d still prefer the one VM, so that your containers aren’t sharing your Proxmox Host’s kernel.

Like LXCs do have a use case, but it sounds like you’re using them to an alternative to regular service containers and that’s not really what it’s for.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 9 months ago (2 children)

    Your statements are surprising to me, because when I initially set this system up I tested against that because I had figured similar.

    My original layout was a full docker environment under a single VM which was only running Debian 12 with docker.

    I remember seeing a good 10gb different with ram usage between offloading the machines off the docker instance onto their own CT's and keeping them all as one unit. I guess this could be chalked down to the docker container implementation being bad, or something being wrong with the vm. It was my primary reason for keeping them isolated, it was a win/win because services had better performance and was easier to manage.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 9 months ago (1 child)

    There are a number of reasons why your docker setup was using too much RAM, including just poorly built containers. You could also swap out docker for podman, which is daemonless and rootless, and registers container workloads with systemd. So if you’re married to the LXCs you can use that for running OCI containers. Also a new version of Proxmox enabled the ability to run OCI containers using LXCs so you can run them directly without docker or podman.

  • source
  • parent
  • hideshow 1 child comment