LocalLLaMA

localllama@sh.itjust.works

5,601 readers

2,905 active

78 users here now

Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.

Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.

As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.

Rules:

Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.

Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.

Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.

Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.

founded 3 years ago

moderation log

moderators

MonsterBug@sh.itjust.works

Smokeydope@lemmy.world

pax@sh.itjust.works

noneabove1182@sh.itjust.works

P100 + Gemma not working properly? (programming.dev)

submitted 2 months ago by CameronDev@programming.dev to c/localllama@sh.itjust.works

11 comments fedilink hide all child comments

Hi All, I just got a P100, and although it appears to work, i cant quite get gemma4 to run on it properly.

My setup is:

Ubuntu 22.04 VM
PCI passthrough the GPU (Connected via thunderbolt) (I know this is unideal, and I might try cut this out later)
Nvidia 580 driver installed.
llama.cpp built from source, with cuda enabled.

I got some mostly normal output from qwen, but whenever I try to load gemma4 I get a lot of <unused49><unused49><unused49><unused49>. I did have it briefly working, no idea what changed, but reboot and its back to unused spam.

Anyone hit this before? Ideas on what I should try change?

you are viewing a single comment's thread
view the rest of the comments

[–] CameronDev@programming.dev [S] 1 point 2 months ago*

Nah, the PSU controls the voltages. All the motherboard does is turn on the PSU by shorting the green PS-ON wire in the mobo cable to ground. The thunderbolt card actually came with a switch to do that, normally I use a paperclip :D.

http://bitsavers.informatik.uni-stuttgart.de/pdf/intel/ATX/ATX_Specification_2.01_199702.PDF

PS-ON is an active low signal that turns on all of the main power rails including 3.3V, 5V, -5V, 12V, and -12V power rails. When this signal is held high by the PC board or left open circuited, outputs of the power rails should not deliver current and should be held at a zero potential with respect to ground. Power should be delivered to the rails only if the PS-ON signal is held at ground potential. This signal should be held at +5VDC by a pull-up resistor internal to the power supply.

Here is an example of a switch: https://a.aliexpress.com/_msxxnjv Mine doesn't have the third wire, I think its just for powering the switch LED though.

permalink

fedilink

source

parent