▲ 6 ▼ Memdoor: A team of 5 Qwen3.8-27B models beats Fable 5 on LiveCodeBench Hard (github.com) submitted 1 week ago by rssbot@lemmy.bestiver.se [B] to c/hackernews@lemmy.bestiver.se comment fedilink hide all child comments Comments
[–] somegeek@programming.dev 1 point 6 days ago (7 children) What is the required hardware? permalink fedilink source hideshow 7 child comments replies: [–] yogthos@lemmy.ml [S] 5 points 6 days ago (6 children) You'd need around 64gb vram to run 27b model at q6/8. You don't have to run multiple models at the same time though, since the same model can work all the different roles sequentially. permalink fedilink source parent hideshow 6 child comments replies: [–] frigge@lemmy.ml 2 points 6 days ago (3 children) I've been running the Ridge-GGUF version on my RX 9070 XT with ~60-80k contextsize with q4 kv cache quantization. Everything fits completely in 16GB VRAM permalink fedilink source parent hideshow 3 child comments replies: [–] yogthos@lemmy.ml [S] 1 point 6 days ago (2 children) Oh that's not bad at all. How do you find q4 performs overall? permalink fedilink source parent hideshow 2 child comments replies: [–] frigge@lemmy.ml 2 points 6 days ago* (1 child) I have nothing really to compare it to in terms of the big ones because i refuse to give anthropic/openai any money. But i have used this qwen model in opencode to port an old CUDA/Optix based project to HIP and that worked fairly well. Not an overly complex refactor to be honest because 99% is just replacing the cuda prefix everywhere with hip, but it did fix the CMakeLists.txt for me and ported the optix prime acceleration structure to hipRT as well. I also let it vibecode a react based portfolio page because i have absolutely 0 knowledge of web development and refuse to learn it because i think it is insane and that turned out perfectly fine as well. So it is absolutely usable for real world projects and the performance is quite OK as well with ~35tok/s. If i up the context size to the full 256k window, the performance drops to 5 tok/s, which is ok, if i let it do a boring refactor over night but at that point it isn't faster than running DeepSeekV4.1-Flash-1017 through Strata and that one is much closer to the current SotA. Definitely exciting times. We are close to being able to run very capable LLMs locally that are more than good enough for development. I am personally not a fan of 100% vibe coded projects anyway, because i like to architect things myself, so letting it go in smaller steps and/or help be with the boring stuff is absolutely fine. Unless the project is some personal stuff that just needs to solve one simple problem then vibe coding is fine imho. permalink fedilink source parent hideshow 1 child comment replies: [–] yogthos@lemmy.ml [S] 2 points 6 days ago Yeah, that sound pretty usable, and seems like making the harness smarter can up the quality quite a bit. And completely agree about architecting things yourself. I find you want to basically do all the scaffolding, and lay out how the project looks like in terms of structure, the call graph, etc., and then let the model fill in the details. But also like you say, for something that's a throwaway, or a small personal project to scratch an itch, it doesn't really matter. permalink fedilink source parent [–] somegeek@programming.dev 1 point 6 days ago (1 child) 🫠 permalink fedilink source parent hideshow 1 child comment replies: [–] magnue@lemmy.world 2 points 6 days ago Money terms somewhere between 3k and 6k permalink fedilink source parent
[–] yogthos@lemmy.ml [S] 5 points 6 days ago (6 children) You'd need around 64gb vram to run 27b model at q6/8. You don't have to run multiple models at the same time though, since the same model can work all the different roles sequentially. permalink fedilink source parent hideshow 6 child comments replies: [–] frigge@lemmy.ml 2 points 6 days ago (3 children) I've been running the Ridge-GGUF version on my RX 9070 XT with ~60-80k contextsize with q4 kv cache quantization. Everything fits completely in 16GB VRAM permalink fedilink source parent hideshow 3 child comments replies: [–] yogthos@lemmy.ml [S] 1 point 6 days ago (2 children) Oh that's not bad at all. How do you find q4 performs overall? permalink fedilink source parent hideshow 2 child comments replies: [–] frigge@lemmy.ml 2 points 6 days ago* (1 child) I have nothing really to compare it to in terms of the big ones because i refuse to give anthropic/openai any money. But i have used this qwen model in opencode to port an old CUDA/Optix based project to HIP and that worked fairly well. Not an overly complex refactor to be honest because 99% is just replacing the cuda prefix everywhere with hip, but it did fix the CMakeLists.txt for me and ported the optix prime acceleration structure to hipRT as well. I also let it vibecode a react based portfolio page because i have absolutely 0 knowledge of web development and refuse to learn it because i think it is insane and that turned out perfectly fine as well. So it is absolutely usable for real world projects and the performance is quite OK as well with ~35tok/s. If i up the context size to the full 256k window, the performance drops to 5 tok/s, which is ok, if i let it do a boring refactor over night but at that point it isn't faster than running DeepSeekV4.1-Flash-1017 through Strata and that one is much closer to the current SotA. Definitely exciting times. We are close to being able to run very capable LLMs locally that are more than good enough for development. I am personally not a fan of 100% vibe coded projects anyway, because i like to architect things myself, so letting it go in smaller steps and/or help be with the boring stuff is absolutely fine. Unless the project is some personal stuff that just needs to solve one simple problem then vibe coding is fine imho. permalink fedilink source parent hideshow 1 child comment replies: [–] yogthos@lemmy.ml [S] 2 points 6 days ago Yeah, that sound pretty usable, and seems like making the harness smarter can up the quality quite a bit. And completely agree about architecting things yourself. I find you want to basically do all the scaffolding, and lay out how the project looks like in terms of structure, the call graph, etc., and then let the model fill in the details. But also like you say, for something that's a throwaway, or a small personal project to scratch an itch, it doesn't really matter. permalink fedilink source parent [–] somegeek@programming.dev 1 point 6 days ago (1 child) 🫠 permalink fedilink source parent hideshow 1 child comment replies: [–] magnue@lemmy.world 2 points 6 days ago Money terms somewhere between 3k and 6k permalink fedilink source parent
[–] frigge@lemmy.ml 2 points 6 days ago (3 children) I've been running the Ridge-GGUF version on my RX 9070 XT with ~60-80k contextsize with q4 kv cache quantization. Everything fits completely in 16GB VRAM permalink fedilink source parent hideshow 3 child comments replies: [–] yogthos@lemmy.ml [S] 1 point 6 days ago (2 children) Oh that's not bad at all. How do you find q4 performs overall? permalink fedilink source parent hideshow 2 child comments replies: [–] frigge@lemmy.ml 2 points 6 days ago* (1 child) I have nothing really to compare it to in terms of the big ones because i refuse to give anthropic/openai any money. But i have used this qwen model in opencode to port an old CUDA/Optix based project to HIP and that worked fairly well. Not an overly complex refactor to be honest because 99% is just replacing the cuda prefix everywhere with hip, but it did fix the CMakeLists.txt for me and ported the optix prime acceleration structure to hipRT as well. I also let it vibecode a react based portfolio page because i have absolutely 0 knowledge of web development and refuse to learn it because i think it is insane and that turned out perfectly fine as well. So it is absolutely usable for real world projects and the performance is quite OK as well with ~35tok/s. If i up the context size to the full 256k window, the performance drops to 5 tok/s, which is ok, if i let it do a boring refactor over night but at that point it isn't faster than running DeepSeekV4.1-Flash-1017 through Strata and that one is much closer to the current SotA. Definitely exciting times. We are close to being able to run very capable LLMs locally that are more than good enough for development. I am personally not a fan of 100% vibe coded projects anyway, because i like to architect things myself, so letting it go in smaller steps and/or help be with the boring stuff is absolutely fine. Unless the project is some personal stuff that just needs to solve one simple problem then vibe coding is fine imho. permalink fedilink source parent hideshow 1 child comment replies: [–] yogthos@lemmy.ml [S] 2 points 6 days ago Yeah, that sound pretty usable, and seems like making the harness smarter can up the quality quite a bit. And completely agree about architecting things yourself. I find you want to basically do all the scaffolding, and lay out how the project looks like in terms of structure, the call graph, etc., and then let the model fill in the details. But also like you say, for something that's a throwaway, or a small personal project to scratch an itch, it doesn't really matter. permalink fedilink source parent
[–] yogthos@lemmy.ml [S] 1 point 6 days ago (2 children) Oh that's not bad at all. How do you find q4 performs overall? permalink fedilink source parent hideshow 2 child comments replies: [–] frigge@lemmy.ml 2 points 6 days ago* (1 child) I have nothing really to compare it to in terms of the big ones because i refuse to give anthropic/openai any money. But i have used this qwen model in opencode to port an old CUDA/Optix based project to HIP and that worked fairly well. Not an overly complex refactor to be honest because 99% is just replacing the cuda prefix everywhere with hip, but it did fix the CMakeLists.txt for me and ported the optix prime acceleration structure to hipRT as well. I also let it vibecode a react based portfolio page because i have absolutely 0 knowledge of web development and refuse to learn it because i think it is insane and that turned out perfectly fine as well. So it is absolutely usable for real world projects and the performance is quite OK as well with ~35tok/s. If i up the context size to the full 256k window, the performance drops to 5 tok/s, which is ok, if i let it do a boring refactor over night but at that point it isn't faster than running DeepSeekV4.1-Flash-1017 through Strata and that one is much closer to the current SotA. Definitely exciting times. We are close to being able to run very capable LLMs locally that are more than good enough for development. I am personally not a fan of 100% vibe coded projects anyway, because i like to architect things myself, so letting it go in smaller steps and/or help be with the boring stuff is absolutely fine. Unless the project is some personal stuff that just needs to solve one simple problem then vibe coding is fine imho. permalink fedilink source parent hideshow 1 child comment replies: [–] yogthos@lemmy.ml [S] 2 points 6 days ago Yeah, that sound pretty usable, and seems like making the harness smarter can up the quality quite a bit. And completely agree about architecting things yourself. I find you want to basically do all the scaffolding, and lay out how the project looks like in terms of structure, the call graph, etc., and then let the model fill in the details. But also like you say, for something that's a throwaway, or a small personal project to scratch an itch, it doesn't really matter. permalink fedilink source parent
[–] frigge@lemmy.ml 2 points 6 days ago* (1 child) I have nothing really to compare it to in terms of the big ones because i refuse to give anthropic/openai any money. But i have used this qwen model in opencode to port an old CUDA/Optix based project to HIP and that worked fairly well. Not an overly complex refactor to be honest because 99% is just replacing the cuda prefix everywhere with hip, but it did fix the CMakeLists.txt for me and ported the optix prime acceleration structure to hipRT as well. I also let it vibecode a react based portfolio page because i have absolutely 0 knowledge of web development and refuse to learn it because i think it is insane and that turned out perfectly fine as well. So it is absolutely usable for real world projects and the performance is quite OK as well with ~35tok/s. If i up the context size to the full 256k window, the performance drops to 5 tok/s, which is ok, if i let it do a boring refactor over night but at that point it isn't faster than running DeepSeekV4.1-Flash-1017 through Strata and that one is much closer to the current SotA. Definitely exciting times. We are close to being able to run very capable LLMs locally that are more than good enough for development. I am personally not a fan of 100% vibe coded projects anyway, because i like to architect things myself, so letting it go in smaller steps and/or help be with the boring stuff is absolutely fine. Unless the project is some personal stuff that just needs to solve one simple problem then vibe coding is fine imho. permalink fedilink source parent hideshow 1 child comment replies: [–] yogthos@lemmy.ml [S] 2 points 6 days ago Yeah, that sound pretty usable, and seems like making the harness smarter can up the quality quite a bit. And completely agree about architecting things yourself. I find you want to basically do all the scaffolding, and lay out how the project looks like in terms of structure, the call graph, etc., and then let the model fill in the details. But also like you say, for something that's a throwaway, or a small personal project to scratch an itch, it doesn't really matter. permalink fedilink source parent
[–] yogthos@lemmy.ml [S] 2 points 6 days ago Yeah, that sound pretty usable, and seems like making the harness smarter can up the quality quite a bit. And completely agree about architecting things yourself. I find you want to basically do all the scaffolding, and lay out how the project looks like in terms of structure, the call graph, etc., and then let the model fill in the details. But also like you say, for something that's a throwaway, or a small personal project to scratch an itch, it doesn't really matter. permalink fedilink source parent
[–] somegeek@programming.dev 1 point 6 days ago (1 child) 🫠 permalink fedilink source parent hideshow 1 child comment replies: [–] magnue@lemmy.world 2 points 6 days ago Money terms somewhere between 3k and 6k permalink fedilink source parent
[–] magnue@lemmy.world 2 points 6 days ago Money terms somewhere between 3k and 6k permalink fedilink source parent
no comments (yet)