▲ 102 ▼ Is there any software that can use it that benefits average user or is it just a waste of silicon??? (i.redd.it) submitted 9 months ago by tatoko556@reddthat.com to c/opensource@programming.dev 30 comments fedilink hide all child comments cross-posted from: https://lemmit.online/post/4242386 This is an automated archive made by the Lemmit Bot. The original was posted on /r/pcmasterrace by /u/trander6face on 2024-10-24 11:11:47+00:00.
[+] sga@piefed.social 21 points 9 months ago (9 children) [deleted] permalink fedilink source hideshow 9 child comments replies: [–] tatoko556@reddthat.com [S] 5 points 9 months ago (3 children) I have heard of ollama before, is this the same thing? permalink fedilink source parent hideshow 3 child comments replies: [–] dyathinkhesaurus@lemmy.world 6 points 9 months ago (1 child) llama.cpp is not the same thing as ollama. It does a similar thing, but better imo. can multiplex/multithread sessions/conversations with llms (where ollama and lmstudio etc. have to queue them up), stuff like that. permalink fedilink source parent hideshow 1 child comment load more comments (1 reply) [–] tatoko556@reddthat.com [S] 4 points 9 months ago I checked, ollama does not use npu permalink fedilink source parent [–] Swedneck@discuss.tchncs.de 2 points 9 months ago (4 children) So, what? Is that only trained on ethical data? Because i bloody doubt it permalink fedilink source parent hideshow 4 child comments replies: [–] Hexarei@beehaw.org 3 points 9 months ago (3 children) It's not a specific model, its a harness for running models. You can find ethically trained models to run on it though iirc. permalink fedilink source parent hideshow 3 child comments replies: [+] sga@piefed.social 1 point 9 months ago (2 children) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] Hexarei@beehaw.org 1 point 9 months ago (1 child) Is it any good? Curious to hear what you're using it for and how it compares permalink fedilink source parent hideshow 1 child comment load more comments (1 reply)
[–] tatoko556@reddthat.com [S] 5 points 9 months ago (3 children) I have heard of ollama before, is this the same thing? permalink fedilink source parent hideshow 3 child comments replies: [–] dyathinkhesaurus@lemmy.world 6 points 9 months ago (1 child) llama.cpp is not the same thing as ollama. It does a similar thing, but better imo. can multiplex/multithread sessions/conversations with llms (where ollama and lmstudio etc. have to queue them up), stuff like that. permalink fedilink source parent hideshow 1 child comment load more comments (1 reply) [–] tatoko556@reddthat.com [S] 4 points 9 months ago I checked, ollama does not use npu permalink fedilink source parent
[–] dyathinkhesaurus@lemmy.world 6 points 9 months ago (1 child) llama.cpp is not the same thing as ollama. It does a similar thing, but better imo. can multiplex/multithread sessions/conversations with llms (where ollama and lmstudio etc. have to queue them up), stuff like that. permalink fedilink source parent hideshow 1 child comment load more comments (1 reply)
[–] tatoko556@reddthat.com [S] 4 points 9 months ago I checked, ollama does not use npu permalink fedilink source parent
[–] Swedneck@discuss.tchncs.de 2 points 9 months ago (4 children) So, what? Is that only trained on ethical data? Because i bloody doubt it permalink fedilink source parent hideshow 4 child comments replies: [–] Hexarei@beehaw.org 3 points 9 months ago (3 children) It's not a specific model, its a harness for running models. You can find ethically trained models to run on it though iirc. permalink fedilink source parent hideshow 3 child comments replies: [+] sga@piefed.social 1 point 9 months ago (2 children) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] Hexarei@beehaw.org 1 point 9 months ago (1 child) Is it any good? Curious to hear what you're using it for and how it compares permalink fedilink source parent hideshow 1 child comment load more comments (1 reply)
[–] Hexarei@beehaw.org 3 points 9 months ago (3 children) It's not a specific model, its a harness for running models. You can find ethically trained models to run on it though iirc. permalink fedilink source parent hideshow 3 child comments replies: [+] sga@piefed.social 1 point 9 months ago (2 children) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] Hexarei@beehaw.org 1 point 9 months ago (1 child) Is it any good? Curious to hear what you're using it for and how it compares permalink fedilink source parent hideshow 1 child comment load more comments (1 reply)
[+] sga@piefed.social 1 point 9 months ago (2 children) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] Hexarei@beehaw.org 1 point 9 months ago (1 child) Is it any good? Curious to hear what you're using it for and how it compares permalink fedilink source parent hideshow 1 child comment load more comments (1 reply)
[–] Hexarei@beehaw.org 1 point 9 months ago (1 child) Is it any good? Curious to hear what you're using it for and how it compares permalink fedilink source parent hideshow 1 child comment load more comments (1 reply)