25
Dump of igdb database or similar? (sh.itjust.works)
submitted 5 months ago by to c/piracy@lemmy.dbzer0.com
11 comments fedilink hide all child comments
 

I am interested in accessing the data of igdb.com for a hobby project. They have an api that is "free", ie it requires a twitch account with 2fa enabled which means i would need to hand over my phone number to twitch. That is a hard no from my side.

Anyone know of a way to get a dump of their database or a similar source where i could get api access without revealing personal information? For my project i am after only title, release year, some sort of rating and some sort of playtime.

you are viewing a single comment's thread
view the rest of the comments
[–] [S] 2 points 5 months ago (3 children)

Ive tried that, but have so far been unable to find a number that has not previously been used for twitch and after a few failed attempts, twitch stops sending out codes.

Ive also experimented with scraping but i am unable to form the request in a way that does not yield 403 for this and howlongtobeat for instance. I did get responses from rawg.io but while it seemed they have a lot of games, stuff like playtime didnt seem too reliable to me (europa univeralis 4 being sub 10hr for example... :p) and ive also yet to find a way to generate all urls to go through all their game pages (urls contain game name, not incremental ids).

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 4 points 5 months ago* (last edited 5 months ago) (1 child)

    The free tier of IGDB has a rate limit of 4 requests per second and they will block you if you try to download the entire database, so this may be a dead end anyway.

    Found a 4 megabyte dump that seems promising though it's from 2018 so outdated.

    https://gist.github.com/LeWawan/5858a9e7bef0f3dc4a79ac8bc1e3380c

    Another one, a 68MB 3 year old "dataset" for AI training that seems related to IGDB.

    https://www.kaggle.com/datasets/anudeepvanjavakam/igdb-api-data/data

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 4 points 5 months ago* (last edited 5 months ago)

    I managed to request 363,020 entries at 500 page size 1 request per second in about 20 minutes, so they don't seem to be super strict. Though weirdly after I load them into a dict to dedupe the IDs I get 362,969, haven't looked into what's going on there.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 5 months ago

    If you have lycamobile or similar in your country... You can walk into a store and just pick one up for free. In some countries they do have an activation fee, but I think that can be as little as €5 in Europe. Other providers offer similar, and then keep the number for 6 months or activate again.

  • permalink
  • fedilink
  • source
  • parent