A simple tool to figure out just how much LLM generated code has been used in a project. Obviously not perfect, but it automates the task of going through commits for "co-authored by" statements and other signs. In the off chance you don't want to use the CLI, there's also a web app!

all 15 comments

sorted by: hot top controversial new old

I already use it. It's GLORIOUS. I'm free from a lot of stress, although not totally free of it. It's better, at least.

  • source
  • [–] 3 points 2 months ago
    [–] 1 point 2 months ago

    That's awesome.

    A way to import a list from ontanimum would be a nice feature.

  • source
  • [+] -7 points 2 months ago (11 children)

    Nobody thinks the output of that thing isn't itself slop?

  • source
  • hideshow 11 child comments
  • [–] 19 points 2 months ago (10 children)

    Looks like a very simple heuristic analysis looking for certain files/commiters. In fact it states in the README that it doesn't go through all the files so it is probably more reliable than those AI "detectors" that also rely on AI.

  • source
  • parent
  • hideshow 10 child comments
  • [–] 14 points 2 months ago* (last edited 2 months ago) (9 children)

    Its just reading the commit history. It doesnt read the files at all. So it only finds overt slop, anyone who does their commits manually would pass completely.

    Edit: https://slopscore.ava.pet/repo/https%3A%2F%2Fgithub%2Ecom%2Fcameroncros%2Fedgy%2Dtable

    This repo is 100% slop (its mine, I know what I did), and it gets marked clean.

  • source
  • parent
  • hideshow 9 child comments
  • [–] 9 points 2 months ago (2 children)

    I mean, realistically speaking that's what I would do manually. So it's just making my life easier by automating it.

    I don't expect it to analyse individual files.

  • source
  • parent
  • hideshow 2 child comments
  • [–] [S] 8 points 2 months ago (1 child)

    Exactly. It can produce false negatives, but by nature of Git's blaming everyone for every change, it makes it (probably) won't produce false positives. It's a give and take in that instance. Do you do a witch hunt and try and track down every LLM generated repo and get some legit repos caught up in an imperfect process. Or do you use actionable evidence to make an informed decision?

    In my opinion, witch hunts are really quite unhealthy for society. When possible, I choose projects with blanket no-AI statements, but when not, this expedites the process of choosing between alternatives. Rustic has less AI generated code than Restic, let's do that. Kitty has hundreds of AI generated commits, let's delete that immediately.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 0 points 2 months ago (4 children)
  • [–] 3 points 2 months ago (3 children)

    Ha, I still hand write most of my code, i used it for that repo to try out local LLMs. Know thy enemy.

    I did hand-polish that code as well, just the initial pass was slopped together.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 2 months ago (2 children)
  • [–] [S] 2 points 2 months ago (1 child)