0
Local coding LLMs without copyleft? (programming.dev)
submitted 5 days ago by to c/programming@programming.dev
14 comments fedilink hide all child comments
 

How can I find a local coding LLM that wasn't trained on copyleft code?

It's an ethical concern, not a legal one, so if the data collection doesn't entirely rule out copyleft, that might be OK

you are viewing a single comment's thread
view the rest of the comments
[–] 4 points 5 days ago (2 children)

Olmo might be the closest you can get

https://allenai.org/olmo (website straight up crashes my phone)

https://huggingface.co/allenai/OLMo-7B

Olmo is trained on the Dolma dataset, “a diverse mixture of web content, scientific papers, code, public-domain books, social media, and encyclopedic materials”, so this might be a more ethical source, but I’m not sure if it’s copyleft free.

https://huggingface.co/papers/2402.00159

You could rent a cluster of GPUs and remove copyleft material and retrain

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] [S] 4 points 5 days ago* (last edited 5 days ago) (1 child)

    Dolma uses starcoder https://huggingface.co/blog/starcoder, which

    are Large Language Models for Code (Code LLMs) trained on permissively licensed data from GitHub

    so this might do. Thanks.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •