[–] 3 points 2 months ago* (last edited 2 months ago) (1 child)

But if you are even a little bit educated as to what the textbooks say about LLMs, they are exactly next-token predictors. There's no ifs-or-buts about it.

Your chess analogy is misinformed too. Your description of neural networks trained on past master games reads like the machine is given pictures of the board and simply learns from that. It also assumes that is how modern day chess engines are written which is not completely true. It is called the concept of "Book" which records past moves in certain positions as a cache. And people don't call that "Next Token Prediction". It's not as sophisticated.. it's simply a lookup table or a cache.

Actually good Chess Engines are search engines walking over a tree or graph of possible next moves. One can argue this is actually a prediction engine... but "Prediction Engine" as a description feels like the machine is rolling the dice far more than a search.. which is the part that's misleading.

Here is my claim: LLMs as they stand today are fundamentally less sophisticated than a Chess Engine since the search part is absent from LLMs. This is not because LLMs are less powerful, no. Their domain is simply different from search engines. Token prediction IS the job description of an LLM. Which is why LLMs are not very good at chess.

  • source
  • [–] 10 points 2 months ago (1 child)

    They are all rightwing conservative companies. They want to dismantle the liberal stronghold in open source... as potentially continuing from project 2025 ish beginnings. First it was Main Stream Media, then social media, and now it's this.

  • source
  • parent
  • context
  • [–] 4 points 2 months ago (5 children)

    Jesus kurwa ja pierdole… Code is code. Who the fuck cares who wrote it? Unless project itself serves racist/transphobic/homophobic/etc purposes (like an app specifically build to doxx gay people or something) I don’t care who’s developing it.

    Yeah but the problem with that argument is that when kind and empathetic people have to starve and beg for money with equal or higher level of quality output compared to those who peddle racism and bigotry, it feels like the world only rewards racism and bigotry.

    You tell me: there are a lot of great projects that don't spread hatred, bigotry and disparity but they don't get even a fraction of the funding this project gets. And this project is just some dotfiles over ArchLinux. Even Archlinux didn't get this much funding in one go.

    The spirit of open-source is built on Kindness and Empathy. When a movement such as that is slowly hijacked by people who are ideologically opposed to kindness and empathy, then the kind and empathetic get deprived of a sense of place and belonging.

  • source
  • parent
  • context
  • [–] 10 points 5 months ago (1 child)

    You expect the people who have been oppressed, labelled terrorists, and silenced to "bring these people into the fold"; The same assholes who bootlick and enable powerful cunts to propagate their vile ways as the only way that works?

  • source
  • parent
  • context
  • [–] -3 points 5 months ago (2 children)

    I feel like the only people purged were the capitalist bootlickers and robber barons. It's always a cooked up story when "Doctors, Teachers, and Businessmen" were purged because you know it was just the last category that felt oppressed even at the slightest course-correction from their vile system.

  • source
  • parent
  • context
  • [–] 5 points 5 months ago (1 child)

    As an ML Acolyte, Even I can tell that it's a well known phenomenon that ensemble methods are more resilient to overfitting issues. What's fascinating is humanity's penchant to forget the past and rediscover stuff.

  • source
  • view more: ‹ prev next ›