you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 1 year ago

If your video card has 16+ Gb of memory you will be able to run it with: NVIDIA cards GTX 10 series or later on ollama.

Ollama is easy, but it leaves a lot of performance on the table.

If you have less than 16 GB, you may be able to get good performance using llama.cpp or especially Ik_llama.cpp.

  • source
  • parent