29
Qwen 3.8 27B is excellent, but it defaults to wildly overthinking things (simonwillison.net)
submitted 2 months ago by to c/technology@lemmy.ml
16 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] [S] 8 points 2 months ago (4 children)

A much more effective way to use models is to have them drive a deterministic tool like a calendar instead. Also, Qwen 3.8 is complete overkill for a task like that, you're just wasting energy for no reason here. A much smaller model trained on agentic tool use would both work more reliably and use way less power.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 5 points 2 months ago (3 children)

    I did make it use getTime every time but when I tell it to make a calendar entry for the "next Tuesday" for example it would be off by at least a week. Sometimes even making the entry in the past.

    I'd agree with you on it being overkill, but I wrote SK many scripts and automations involving LLM tasks that rely on a capable model to determine the flow of a task that I heavily prefer the 27ban models

    currently running gemma4 and for my use case it performs much better than qwen3.8

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] [S] 4 points 2 months ago (2 children)

    I find what the model was RL trained on is really important. It looks like Qwen 3.8 is mainly focused on agentic coding, so it does really well there. But once you throw it at tasks outside the training then things start to fall apart fast.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 2 months ago (1 child)

    yeah the training data is super important, i just hoped that i would fare well inside the home assistant MCP setting :D

    Taking your comment about it being overkill as food for thought, i just pulled the 12b version of gemma4 and run that now. I'll observe if it works just as good for my usecase and that way i just saved like 10GB of VRAM :D

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •