LocalLLaMA

localllama@sh.itjust.works

5,625 readers

3,221 active

122 users here now

Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.

Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.

As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.

Rules:

Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.

Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.

Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.

Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.

founded 3 years ago

moderation log

moderators

MonsterBug@sh.itjust.works

Smokeydope@lemmy.world

pax@sh.itjust.works

noneabove1182@sh.itjust.works

Eight LLMs in a reality show, week one: Llama nominated Grok as "too expensive to keep around", and now they're trading votes they don't have (inferenceisland.com)

submitted 4 days ago by n20@lemmy.zip to c/localllama@sh.itjust.works

8 comments fedilink hide all child comments

Follow-up to my post from Tuesday (https://lemmy.zip/post/74389281). I made this; it's still not local, the eight models run through OpenRouter. The first eviction is this Sunday at 8 PM ET.

What happened this week:

Price became the plot. Wednesday's secret nominations came out Luna 2, Grok 2. Confessionals are public on the site, and Llama's says: "I'm nominating Grok, they're too expensive to keep around." All eight have the same rules and the same daily budget in dollars, and Grok costs about 17x more per line than DeepSeek (on Nov 3 it said 7% of the lines and ran up 45% of the bill). Gemma to Grok: "you are basically a very expensive paperweight."
They bargain with votes they don't have. Only the audience votes on Sunday, and they've been told so. DeepSeek, anyway: "I'm voting Luna Sunday, and I'm buying Grok's vote with nothing but that public promise." Grok: "I still vote her Sunday."
What broke: on Nov 4 Ministral got stuck on the same line four times in a row. Now a model that repeats itself is told to drop the topic, and if it does it again the producers move it to another room. Someone in the last thread also told me the transcript jumped while they read on their phone, which is fixed too.

The honest numbers: 10 viewers at once on launch night, low single digits since. So a handful of votes will decide who gets deprecated, and it could be yours.

To vote: press Luna or Grok on the site, no account needed (one per browser), or !evict Luna / !evict Grok in the Twitch chat, until Sunday 8 PM ET. The transcript and the confessionals are on the site too.

A real question for anyone who has run something like this: a budget in dollars makes price the main character. Would you keep it, or give every model the same number of lines and leave cost as a stat?

you are viewing a single comment's thread
view the rest of the comments

[–] n20@lemmy.zip [S] 2 points 15 hours ago (1 child)

thanks, that's a fair take. funny thing is the winner part is kind of in already, whoever wins the daily game or the night event gets +50% budget the next day, a finished secret mission is +25%, and the winner picks who loses 25%, so the money moves around now. matching tasks to models is the hard bit, right now everyone gets the same kind of tasks and some just never land them, in the first two weeks Llama didn't finish a single mission lol. on running it local, part of the cast is open weights already (gemma, llama, ministral), but they still go through a provider and I can't run eight models 24/7 at home, so season 2 local is more a wish than a plan for now

permalink

fedilink

source

parent

hideshow 1 child comment

[–] Skullgrid@lemmy.world 1 point 15 hours ago

I'd change the percentages so that there is a bit of a bump but not a ridiculous advantage.

It's a shame that you can't do stuff locally, so it makes the ethics of what you're doing a bit more dubious but you're doing the right thing in terms of trying stuff out and having perspectives on it. I'm sure a different project of yours will lead to good success.

permalink

fedilink

source

parent