you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 6 months ago (2 children)

depends on your hardware and your preferred language. i think wizardcoder is a pretty common choice but the smallest useful version is around 14GB so you need the vram to accommodate it.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 6 months ago (1 child)

    Thanks, I'll dig into this. BTW, I have a 9070 XT, with 16 Gb VRAM, so it should do the job I guess.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 6 months ago

    you need space for the context and runtime parameters too, but i think it should work. worst case there are some offloading settings you can do depending on the server you use. only way to knew is to try, really.

  • source
  • parent