211
The circle of life (lemmy.ml)
submitted 5 months ago by to c/programmerhumor@lemmy.ml
10 comments fedilink hide all child comments
 

all 10 comments

sorted by: hot top controversial new old
[–] 64 points 5 months ago (1 child)

This would be a fantastic time to try to organize labor in tech. We have needed unions badly for decades now. I hope the AI debacle coupled with rising socialist sentiments in the face of global fascism can finally give us the momentum to get there.

  • permalink
  • fedilink
  • source
  • hideshow 1 child comment
    •
    • [–] 7 points 5 months ago

    Demand a raise.

  • permalink
  • fedilink
  • source
    •
    [–] 4 points 5 months ago (1 child)

    Why does Jim look odd? Does he have a different face?

  • permalink
  • fedilink
  • source
  • hideshow 1 child comment
    •
    • [–] 3 points 5 months ago

    Gondor calls for aid

  • permalink
  • fedilink
  • source
    •
    [–] 5 points 5 months ago (3 children)

    One jas to Knox how expensive LLMs are by trying to run one at home. Tried running it myself, figured out I need more than 128GiB of RAM and thousand dollars in grahics card. Figured out a $5 openai per month is cheaper, and also understood they are burning money by providing a free service and 20$ subscription.

  • permalink
  • fedilink
  • source
  • hideshow 3 child comments
    •
  • [–] [S] 7 points 5 months ago (2 children)

    You should be able to get very decent performance with 128gb vram running Qwen 3.6 with something like https://github.com/itigges22/ATLAS especially if you run MTP https://huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF

    A friend of mine gets something like 50 tokens a second with it, and output quality is quite decent.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 5 months ago (1 child)

    How does it compare to largest deepseek ans Claude opus 4.6? I hot used to blazing fast speed and accurate results. I'm not buying a server and 128 GB of RAM just to run a model similar to gpt-4.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] [S] 2 points 5 months ago

    ATLAS has some benchmarks in the repo, and it's comparable to opus 4.6, you don't actually even need 128gb model for that. An 8 bit quantized model will run with around 32gb and still perform quite well.

  • permalink
  • fedilink
  • source
  • parent
    • 139 comments in c/programmer_humor@programming.dev view that post