You can get by surprisingly well on 20b parameter models using a Mac with decent ram or even 8b parameter models that fit on most high end (eg 16gb) model cards. Depends on your use cases but I almost exclusively use smaller local models.
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
load more comments
(1 reply)