Given how memory is the bottleneck especially at the very low end it makes me wonder if one bit quantization of an extremely large model would be a gigabyte per gigabyte of ram better
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments