you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 4 months ago

Actually, training a very big model on trillions of tokens using huge clusters is one way of doing that, but certainly not the only way.

Communities have been fine-tuning models on much smaller machines and on a budget of hundreds of bucks.

Right now, money is abundant and feeds even open on its models, so it would be stupid to not go in that direction. But when it dries up, it won't be the end of models improving.

  • source
  • parent