393
Huggingface have scraped the entirety of Github. (huggingface.co)
submitted 3 months ago by to c/programming@programming.dev
108 comments fedilink hide all child comments
 

Would be a terrible shame if lots of people opted out.

If you are a EU citizen you might also want to write a complaint to privacy@huggingface.co because they are collecting your personally identifiable information in machine-readable form which they are distributing to third parties.

you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 3 months ago

From the linked webpage readme:

2.1 Classify. Each file is labelled permissive (at least one permissive license detected, no conflicting non-permissive license), no_license (no licenses detected, or only non-license legal texts such as CLAs), or non_permissive. The permissive allowlist follows the Blue Oak Council list plus licenses categorized as Permissive or Public Domain by ScanCode. Files classified as non_permissive are excluded from both released datasets.

From https://www.bigcode-project.org/docs/about/the-stack/:

v1.1: The three copyleft licenses (MPL/EPL/LGPL) were excluded and the list of permissive licenses extended to 193 licenses in total. The list of programming languages was increased from 30 to 358 languages. Also opt-out request submitted by 15.11.2022 were excluded from this ersion of the dataset. The resulting near-deduplicated dataset is 6TB in size.

So MPL/EPL/LGPL are already not part of the dataset.

So… why were they in there? Was this added for v1.1?

"one permissive license" - So if my project includes a lib and I include the license file for that for the license notice…?

  • permalink
  • fedilink
  • source
    •