you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 2 months ago (5 children)

Having issues with it (Unsloth's gguf).

0.27.136.404 I srv  proxy_reques: proxying request to model unsloth/Qwen3.8-27B-GGUF:Q4_K_M on port 55741
[55741] 0.14.424.772 W srv    operator(): got exception: {"error":{"code":500,"message":"\n------------\nWhile executing CallExpression at line 106, column 32 in source:\n...first %}↵            {{- raise_exception('System message must be at the beginnin...\n                                           ^\nError: Jinja Exception: System message must be at the beginning.","type":"server_error"}}

  • source
  • hideshow 5 child comments
  • [–] 7 points 2 months ago* (3 children)

    Got an answer - there were indeed issue with some quants

    Actually edit - UD-Q4_K_XL and all UD-* has our corrected chat teaplate

    We need to update the non UD-* ones with our chat template - stay tuned

  • source
  • parent
  • hideshow 3 child comments