T156 comments on GitHub just switched Copilot to metered billing, and developers are watching months of credits vanish in a single day

Technology

89,574 readers

13,778 active

2,336 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 3 years ago

moderators

[–] T156@lemmy.world 6 points 5 months ago (2 children)

Ever run an AI model locally? If you want the most capability you need a fast GPU with 32-48gb RAM. And that's all for you, ONE user.

Even then, that's quite small. Top of the line frontier models would be looking at hundreds of gigabytes of video memory, and just as much RAM.

A terabyte of VRAM/RAM needed for something like CoPilot is probably a fairly sensible estimate.

permalink

fedilink

source

parent

hideshow 2 child comments

[–] phx@lemmy.world 9 points 5 months ago (1 child)

Depends on what you want to do, the model, and optimization or quantization.

A lot of LLM stuff that seemed pretty amazing a few years ago - chatbots and the like that respond to questions in plain language - can run in comparatively light hardware. Coding agents can take more, but could also be optimized against a particular language and spit out useful snippets.

Image stuff can be pretty complex especially at higher resolutions and detail, and creating seamless video segments gets expensive on hardware, fast.

permalink

fedilink

source

parent

hideshow 1 child comment

[–] SirEDCaLot@lemmy.today 3 points 5 months ago

Quite true. The thing is, there aren't billions and billions of dollars in chatbots. The billions are for the creative stuff and the code.

And that is where the reckoning / correction will come from, the bill has to come due eventually. When top end generative AI starts to have a real cost associated with it, then it's no longer a blanket 'everyone start using this immediately' mandate, it prompts some consideration of cost versus output quality.

permalink

fedilink

source

parent