Yet, even when heavily compressed, it requires roughly 240GB of memory just to load.
Ah I'll just pop it in the ol' Raspberry Pi then, easy peasy.
Yet, even when heavily compressed, it requires roughly 240GB of memory just to load.
Ah I'll just pop it in the ol' Raspberry Pi then, easy peasy.
Sucks that people lump AI into a single category of whatever cloud-hosted subscription that tech bros from Silicon Valley are pushing.
Given how memory is the bottleneck especially at the very low end it makes me wonder if one bit quantization of an extremely large model would be a gigabyte per gigabyte of ram better
all 15 comments