▲ 512 ▼ Other AI Companies Should Be Very Afraid of What Just Happened to Character.AI, According to Its Own Bankruptcy Documents: Be afraid. (futurism.com) submitted 3 weeks ago by eicker@lemmy.world to c/technology@lemmy.world 156 comments fedilink hide all child comments
[–] Pratai@piefed.ca 5 points 3 weeks ago (6 children) Anyone who lacks the skill to to it themselves… permalink fedilink source parent hideshow 6 child comments replies: [–] Wildmimic@anarchist.nexus 3 points 3 weeks ago (5 children) There's not much skill to be had there - It is literally just downloading an app nowadays if you do not want to do anything special, especially on Linux (Newelle is great stuff!). But I am sure there are similar packages available for Windows too. permalink fedilink source parent hideshow 5 child comments replies: [–] jay@pie.zerojay.com 2 points 3 weeks ago (4 children) Excuse my ignorance - I don't have much experience with AI to this point - but what does Newelle have that something like Unsloth or LMStudio does not? What makes it great stuff? permalink fedilink source parent hideshow 4 child comments replies: [–] Wildmimic@anarchist.nexus 3 points 3 weeks ago* (3 children) honestly? ease of use. I like Unsloth too, but Newelle is really an easy solution if you just want to use LLMs on you own hardware (or use inference providers) that requires practically no knowledge at all - it is focused on providing something to use LLMs without anything else interfering. Unsloth does well too, but if i hadn't learnt as much as i already did when i started using it, i would probably have been overwhelmed (outside of it crashing pathetically when following symlinks for local model selection - you can define these directories, but the quick selection then just makes it crash) permalink fedilink source parent hideshow 3 child comments replies: [–] jay@pie.zerojay.com 2 points 3 weeks ago (2 children) Thanks, I appreciate you taking the time to answer. permalink fedilink source parent hideshow 2 child comments replies: [–] Wildmimic@anarchist.nexus 3 points 3 weeks ago* (last edited 3 weeks ago) (1 child) Np. I actually use Newelle with an fork of llama.cpp that introduces the ability to use ternary models - these work a lot better under tight memory constraints, and achieve performance similar to models with 2-3 times the parameters. Was as easy as installing the fork, starting it and pointing Newelle there instead of the integrated llama.cpp. You can do that with Unsloth too of course, but it's not that smooth. (if you are interested: https://huggingface.co/prism-ml/collections for the ternary models, https://github.com/PrismML-Eng/llama.cpp for the build that allows for ternary models.) permalink fedilink source parent hideshow 1 child comment replies: [–] jay@pie.zerojay.com 1 point 3 weeks ago Thanks, I appreciate it. Had not heard of these before. permalink fedilink source parent
[–] Wildmimic@anarchist.nexus 3 points 3 weeks ago (5 children) There's not much skill to be had there - It is literally just downloading an app nowadays if you do not want to do anything special, especially on Linux (Newelle is great stuff!). But I am sure there are similar packages available for Windows too. permalink fedilink source parent hideshow 5 child comments replies: [–] jay@pie.zerojay.com 2 points 3 weeks ago (4 children) Excuse my ignorance - I don't have much experience with AI to this point - but what does Newelle have that something like Unsloth or LMStudio does not? What makes it great stuff? permalink fedilink source parent hideshow 4 child comments replies: [–] Wildmimic@anarchist.nexus 3 points 3 weeks ago* (3 children) honestly? ease of use. I like Unsloth too, but Newelle is really an easy solution if you just want to use LLMs on you own hardware (or use inference providers) that requires practically no knowledge at all - it is focused on providing something to use LLMs without anything else interfering. Unsloth does well too, but if i hadn't learnt as much as i already did when i started using it, i would probably have been overwhelmed (outside of it crashing pathetically when following symlinks for local model selection - you can define these directories, but the quick selection then just makes it crash) permalink fedilink source parent hideshow 3 child comments replies: [–] jay@pie.zerojay.com 2 points 3 weeks ago (2 children) Thanks, I appreciate you taking the time to answer. permalink fedilink source parent hideshow 2 child comments replies: [–] Wildmimic@anarchist.nexus 3 points 3 weeks ago* (last edited 3 weeks ago) (1 child) Np. I actually use Newelle with an fork of llama.cpp that introduces the ability to use ternary models - these work a lot better under tight memory constraints, and achieve performance similar to models with 2-3 times the parameters. Was as easy as installing the fork, starting it and pointing Newelle there instead of the integrated llama.cpp. You can do that with Unsloth too of course, but it's not that smooth. (if you are interested: https://huggingface.co/prism-ml/collections for the ternary models, https://github.com/PrismML-Eng/llama.cpp for the build that allows for ternary models.) permalink fedilink source parent hideshow 1 child comment replies: [–] jay@pie.zerojay.com 1 point 3 weeks ago Thanks, I appreciate it. Had not heard of these before. permalink fedilink source parent
[–] jay@pie.zerojay.com 2 points 3 weeks ago (4 children) Excuse my ignorance - I don't have much experience with AI to this point - but what does Newelle have that something like Unsloth or LMStudio does not? What makes it great stuff? permalink fedilink source parent hideshow 4 child comments replies: [–] Wildmimic@anarchist.nexus 3 points 3 weeks ago* (3 children) honestly? ease of use. I like Unsloth too, but Newelle is really an easy solution if you just want to use LLMs on you own hardware (or use inference providers) that requires practically no knowledge at all - it is focused on providing something to use LLMs without anything else interfering. Unsloth does well too, but if i hadn't learnt as much as i already did when i started using it, i would probably have been overwhelmed (outside of it crashing pathetically when following symlinks for local model selection - you can define these directories, but the quick selection then just makes it crash) permalink fedilink source parent hideshow 3 child comments replies: [–] jay@pie.zerojay.com 2 points 3 weeks ago (2 children) Thanks, I appreciate you taking the time to answer. permalink fedilink source parent hideshow 2 child comments replies: [–] Wildmimic@anarchist.nexus 3 points 3 weeks ago* (last edited 3 weeks ago) (1 child) Np. I actually use Newelle with an fork of llama.cpp that introduces the ability to use ternary models - these work a lot better under tight memory constraints, and achieve performance similar to models with 2-3 times the parameters. Was as easy as installing the fork, starting it and pointing Newelle there instead of the integrated llama.cpp. You can do that with Unsloth too of course, but it's not that smooth. (if you are interested: https://huggingface.co/prism-ml/collections for the ternary models, https://github.com/PrismML-Eng/llama.cpp for the build that allows for ternary models.) permalink fedilink source parent hideshow 1 child comment replies: [–] jay@pie.zerojay.com 1 point 3 weeks ago Thanks, I appreciate it. Had not heard of these before. permalink fedilink source parent
[–] Wildmimic@anarchist.nexus 3 points 3 weeks ago* (3 children) honestly? ease of use. I like Unsloth too, but Newelle is really an easy solution if you just want to use LLMs on you own hardware (or use inference providers) that requires practically no knowledge at all - it is focused on providing something to use LLMs without anything else interfering. Unsloth does well too, but if i hadn't learnt as much as i already did when i started using it, i would probably have been overwhelmed (outside of it crashing pathetically when following symlinks for local model selection - you can define these directories, but the quick selection then just makes it crash) permalink fedilink source parent hideshow 3 child comments replies: [–] jay@pie.zerojay.com 2 points 3 weeks ago (2 children) Thanks, I appreciate you taking the time to answer. permalink fedilink source parent hideshow 2 child comments replies: [–] Wildmimic@anarchist.nexus 3 points 3 weeks ago* (last edited 3 weeks ago) (1 child) Np. I actually use Newelle with an fork of llama.cpp that introduces the ability to use ternary models - these work a lot better under tight memory constraints, and achieve performance similar to models with 2-3 times the parameters. Was as easy as installing the fork, starting it and pointing Newelle there instead of the integrated llama.cpp. You can do that with Unsloth too of course, but it's not that smooth. (if you are interested: https://huggingface.co/prism-ml/collections for the ternary models, https://github.com/PrismML-Eng/llama.cpp for the build that allows for ternary models.) permalink fedilink source parent hideshow 1 child comment replies: [–] jay@pie.zerojay.com 1 point 3 weeks ago Thanks, I appreciate it. Had not heard of these before. permalink fedilink source parent
[–] jay@pie.zerojay.com 2 points 3 weeks ago (2 children) Thanks, I appreciate you taking the time to answer. permalink fedilink source parent hideshow 2 child comments replies: [–] Wildmimic@anarchist.nexus 3 points 3 weeks ago* (last edited 3 weeks ago) (1 child) Np. I actually use Newelle with an fork of llama.cpp that introduces the ability to use ternary models - these work a lot better under tight memory constraints, and achieve performance similar to models with 2-3 times the parameters. Was as easy as installing the fork, starting it and pointing Newelle there instead of the integrated llama.cpp. You can do that with Unsloth too of course, but it's not that smooth. (if you are interested: https://huggingface.co/prism-ml/collections for the ternary models, https://github.com/PrismML-Eng/llama.cpp for the build that allows for ternary models.) permalink fedilink source parent hideshow 1 child comment replies: [–] jay@pie.zerojay.com 1 point 3 weeks ago Thanks, I appreciate it. Had not heard of these before. permalink fedilink source parent
[–] Wildmimic@anarchist.nexus 3 points 3 weeks ago* (last edited 3 weeks ago) (1 child) Np. I actually use Newelle with an fork of llama.cpp that introduces the ability to use ternary models - these work a lot better under tight memory constraints, and achieve performance similar to models with 2-3 times the parameters. Was as easy as installing the fork, starting it and pointing Newelle there instead of the integrated llama.cpp. You can do that with Unsloth too of course, but it's not that smooth. (if you are interested: https://huggingface.co/prism-ml/collections for the ternary models, https://github.com/PrismML-Eng/llama.cpp for the build that allows for ternary models.) permalink fedilink source parent hideshow 1 child comment replies: [–] jay@pie.zerojay.com 1 point 3 weeks ago Thanks, I appreciate it. Had not heard of these before. permalink fedilink source parent
[–] jay@pie.zerojay.com 1 point 3 weeks ago Thanks, I appreciate it. Had not heard of these before. permalink fedilink source parent