[–] 6 points 3 months ago* (4 children)

My actual router has been running OpenWRT for a long time now, but I really ought to put it on my wireless access points, too.

I also ought to finally look into OpenWISP (which, as I understand it, is a Free Software alternative to things like Omada or UniFi).

  • source
  • [–] 4 points 3 months ago

    If you're running on AMD hardware, drop the power cap to save energy. (e.g. by poking values into /sys/class/drm/card1/device/hwmon/.../power1_cap or similar; check the min and max files adjacent to it for the ranges.) I can lower the power usage by 100W on a discrete AMD GPU and only lose about ~3% tok/s generation (inference is limited more by memory bandwidth than by compute typically on modern hardware). Test different settings on your actual hardware to find the sweet spot balancing performance vs energy usage!

  • source
  • [–] 1 point 3 months ago

    Yeah I run the Gemma 4 12B QAT on my homeserver. It has a 12GB GPU (Nvidia A2000) and that model gives the most bang for the buck being both fast and multimodal.

  • source
  • [–] 3 points 5 months ago

    This doesn't seem to take into account CPU MoE, which can make a huge amount of difference - Running a bigger MoE model is better than a small model that fits in your GPU if you have the CPU resources. I run Qwen3.6 (the 30b/e4b version) in MoE at around 40 tok/s on my 5070+Ryzen 9 5950X, and it's way better than that tool's recommended 9b.

  • source
  • [–] 2 points 5 months ago

    This feels useless. At least for homelabbers, ollama's model page tells us more useful info. And if a newbie goes there they'll be misguided.

    Also, there's a lot of people who use CPUs, they don't list anything about them at all. Like I cannot fit Gemma 4 on my GPU, but ollama offloads it to CPU, and even with small GPUs you can get good performance.

    And for nearly all small models, it recommends RTX 5060. Which is a very stupid choice.

  • source