all 7 comments

sorted by: hot top controversial new old
[–] 2 points 1 year ago*

With sparse attention, very interesting. It seems GQA is a thing of the past.

I especially love Deepseek's 'public research' aspect: they trained this and Terminus the same way, so the attention schemes are (more-or-less) directly comparable. That's awesome.

GLM 4.6 is reportedly about to drop too. Which is great, as 4.5 is without a doubt my daily driver now.

  • source
  • [–] -4 points 1 year ago (5 children)

    New version of the propaganda machine dropped 🤦‍♂️

  • source
  • hideshow 5 child comments
  • [–] 7 points 1 year ago* (4 children)

    Deepseek is only bad via the chat app, and whatever prefilter (or finetune?) they censor it with.

    The model itself (via API or run locally) isn't too bad, especially with a system prompt or completion syntax to squash refusals. Obviously there are CCP mandated gaps (which you can just add in via context), but it's not as tankie as you'd think.

  • source
  • parent
  • hideshow 4 child comments
  • [–] [S] 3 points 1 year ago (3 children)

    Just ignore them on anything AI related, they are the polar opposite of the AI Tech Bros. Shitting on anything and everyone using AI in any form for anything

  • source
  • parent
  • hideshow 3 child comments
  • 2 comments in c/technology@lemmy.ml view that post
    [–] 2 points 1 year ago (1 child)