Well a local model responding to a prompt on less than 20gb of vram (gaming computer) costs less power than booting up any recent AAA high fps game. The primary cost in power is r&d. Training the next model to be "the best" is an Arms race. 90% of power consumption is trying to train the next model in 100 different ways. China was able to build off of chat cpt tech and built a model that has similar abilities and smartness for only 5 million. I think I won't update my local model until there is actually more abilities in the next one.
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
replies: