you are viewing a single comment's thread
view the rest of the comments
[–] 14 points 3 months ago (2 children)

Same, getting ~3 trillion parameters in consumer hardware is rough.

If Nvidia has any foresight they'll see the writing on the wall and start getting higher memory Spark style SMB inference machines, few people in the long run are going to pay retail API token costs,

  • source
  • parent
  • hideshow 2 child comments