1054
Anthropic logged net loss of over $80 billion in first half of 2026 (www.reuters.com)
submitted 2 weeks ago by to c/technology@lemmy.world
311 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 22 points 2 weeks ago (23 children)

I just recently installed Strata which runs on qwen 3.8 flash next. I'm getting over 50 toks. Now a deepseek v4 version is out along with glm 5.3 flash. It's slower but useable and very capable. All this on a 5080 16GB and 128GB ddr5.

The future for me is going to be open weights and on your own hardware.

  • permalink
  • fedilink
  • source
  • hideshow 23 child comments
    •
  • [–] 14 points 2 weeks ago (2 children)

    That's how it needs to be, and how it should be. Otherwise, we are all permanently screwed by big tech, subscribed and licensed to even own a computer by 2035; MicroSlop says open-source AI is good somehow, but we will see through their actions if they lock down their new line of NVIDIA Spark laptops to an approved number of models. I heard rumours that Deekseek might be banned in America; I can't see how that would work though.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 weeks ago (1 child)

    Deepseek is open weights. You can just download and use it locally. I would never run deepseek using their servers. Or z.ai servers. That I agree with China is still pretty bad.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 2 weeks ago* (last edited 2 weeks ago)

    While I have no problem with most of China's principles regarding business. I don't know about all their politics and laws, nor have I given them much thought, but they deserve more respect than they get for essentially being the whole world's industrial estate and being able to pull off the amazing things they do while having virtually no copyright laws. Open-sourcing so many things we would rather make proprietary, thus very expensive for ourselves. I'm grateful to them for deepseek! Even if they want to spy on me a little, at least I know to expect it lol =)

  • permalink
  • fedilink
  • source
  • parent
  • [–] 7 points 2 weeks ago (4 children)

    I mean that sounds great and I'm happy for you, but you're working with hardware that costs more than most people on here's cars when it's available at all. How viable would your workflow be on an 8gb 1080ti with 32gb of ddr4?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 2 weeks ago (2 children)

    I mean, it's not really a toy. It's a business expense. I use it everyday to make money.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 weeks ago (1 child)

    At no point did I say it was man. Just not something most people have access to. "Got to have money to make it" painfully true as usual. I have $234 in my wallet, a late 90s Honda to crash in, and sheer force of will to try and change that (it's not working lol). The 32GB/1080ti example is the nicest computer any of my friends own, my own rig is a scavenged parts pile centered around a 10th gen i7 that will be complete when some ram falls into my lap and right now I'm posting from a phone that was $150 6 years ago.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 2 weeks ago

    You know, that's fair. Most people do not need a rig like mine. I would still be using my 8 year old computer if my new clients data wasn't so sensitive that I have to run local inference.

    As someone who was also broke and lived in an old Honda, I know what you're going through and I've been there. If you truly want to change your outcome, you gotta find a job that has high growth potential and a low barrier for entry. Personally I worked at a bank for almost a decade. Started as a teller(low barrier,no college needed), ended as a regional manager(high growth). I hated every minute of it. But it gave me a chance to save up and find a career I enjoy.

    Good luck my dude. You will be successful. I believe in you.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 3 points 2 weeks ago (3 children)

    Yeah but getting your own hardware is getting expensive too.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 2 points 2 weeks ago (2 children)

    Only because there is artificial scarcity in play

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 weeks ago (1 child)

    There's plenty of actual scarcity to go around too

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 2 weeks ago

    I mean, besides oil prices, the only reason why hardware is more expensive than it was before is because people are hoarding it, fearing that it won't be available in the future. But there's no reason why new and better hardware shouldn't be engineered and made available unless there is another world war.

    The electronic hardware cartel has stopped designing new hardware, but it was a deliberate choice to inflate prices. They even backpedaled on 5090s

  • permalink
  • fedilink
  • source
  • parent
  • [–] 5 points 2 weeks ago

    Nice setup. I'm running Qwen3.8 27B at 17GB with a 32k context window on a 3090. It does most of what I need for everyday use and its all local, private, no internet connection needed.

    The gap between running an LLM locally and cloud API is closing fast.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 2 weeks ago (7 children)

    How do you run qwen 3.8 flash next on 16gb vram, if you don't mind me asking ?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [–] 5 points 2 weeks ago (4 children)

    At 3 tokens a second and calling it useble I would guess?

    I see people claiming that shit all the time and it's baffling.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 2 points 2 weeks ago (2 children)

    They said 50- i looked up strata, and it uses smaller quantizations/optimizations of that model depending on your system.

    I have 48g vram and 192g system ram and get ~15t/s on qwen 3.8 flash next at Q4. Off the top of my head. I think Q6 is 12.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 weeks ago

    need 64gb of system ram. qwen next architecture uses smaller experts than others, and it can keep up with ram swap. I believe it uses a file on disk (ngram) to predict what experts will be needed.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 2 weeks ago (1 child)

    Running on your hardware is always going to be less efficient than running in a data center. You can run small models on green data centers - best of both worlds (unless you need privacy then run local).

    Ever since I switched to Cortecs I'm spending like 20 cents / month for my LLM usage cuz I mostly use small models and try using them responsibly.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •