In a comment over at !selfhosted@lemmy.world someone wrote:

Containers are not a security barrier. They cannot be, they never will be.

I'm no expert but my mental model of containers was different in that I thought containerized processes live in somehow separate environments, at least filesystem- and network-wise.

I run self-hosted services in containers on my server to isolate them from each other, and now and then binaries I don't fully trust (things like suspected slop code or not well known python libraries) in containers on my laptop.

Does someone with more technical knowledge about how containers work feel like explaining whether that is a bad idea and if so why? That would be highly appreciated.

Edit: missing words and punctuation.

Edit 2: I would like to thank everyone who took the time to reply. My key take aways are that I will move my self-hosted services into VMs for proper isolation, but that it's probably fine to use a container when I want to make sure an LLM doesn't accidentally rm -rf my laptop system when it gets stuck.

Basically the title.

For me, a dense model around 20b, with an additional engram table of 10-20b, or a sparse MoE around 80b would be amazing.

I didn't even know they train their own models.

Have been playing around with this on my laptop's Radeon 780M, with 16GB of RAM assigned to the GPU. llama.cpp with Vulkan reaches a comfortable 30t/s TG with up to 1000t/s PP when running Q8_0 with unquantized KV-cache.

In the few rather simple summarisation tasks I gave it so far it performed better than expected, but of course it's very small MoE and you shouldn't expect too much. Looking forward to evaluating this in Pi on small agentic coding tasks.

Have some of you been playing around with it?

Some days ago I saw people who attended a Fridays for Future demonstration excitedly put political stickers on a shiny blue Lamborghini which was obviously parked at the wrong point in spacetime.

When discussing this with a friend, we concluded that there was quite strong symbolism in that situation - like direct payback for the unnecessary pollution of the planet, the car being the canvas where the activists were able to project their anger onto.

We also talked about luxury cars being a symbol of social inequality.

And only later it hit me, how luxury cars, among other things, are usually called status symbols and how actually they could also be called symbols of equality.

From the model card, sounds interesting:

The "Unified" in Gemma 4 12B Unified refers to its encoder-free architecture. Other Gemma 4 models use dedicated encoders to process multimodal data before passing it to the LLM. Gemma 4 12B eliminates these encoders entirely, projecting raw image patches and audio waveforms directly into the LLM's embedding space through lightweight linear layers. This unified approach means all modalities flow straight into a single decoder-only transformer, reducing multimodal latency and allowing the entire model to be fine-tuned in one pass.

The benchmarks put it closer to the 26b MoE than to the E variants of the Gemma4 series, but mostly below Qwen3.5 9b.

Looking forward to giving it a shot.

WTF? Do you see this as well?

In case you missed it, 2-3 weeks ago, experimental tensor-parallelism support was merged into llama.cpp.

In a nutshell, this allows in multi-GPU setups to not only combine the VRAM of the cards but also their computing power. The results depend a lot on the specific setup and model, but on my 3x RTX 2000e Ada rig running Qwen3.6-35b it almost doubled generation throughput (these are low-powered cards which are not very powerful on their own).

The option to turn it on is --split-mode tensor.

It's not yet officially documented, I assume because it's still experimental. But since #22362 was merged yesterday, in my case it now also work for the latest Qwen3.6 models.

The wait is over, most ggufs are already up. Nice to see there's models for many different hardware configurations.

It's out!

Hey everyone! I was just skimming through some inference benchmarks of other people and noticed the driver version is usually mentioned. It made me wonder how relevant this is. My prod server runs Debian 12 so the packaged nvidia drivers are rather old, but I'd prefer not to mess with the drivers if it won't bring a benefit. Does any of you have any experience or did do some testing?