I run llms using a 780m you'll be fine. I get pretty close to 10 tokens a second for larger 20B+ models.
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I run llms using a 780m you'll be fine. I get pretty close to 10 tokens a second for larger 20B+ models.