It’s the smallest unit an LLM can process. If you write a long florid prompt, it’s more tokens and therefore costs more for the LLM to interpret. There’s literally a “caveman” mod where it makes the LLM respond in caveman-speak, which means loss tokens and therefore cheaper. What’s odd (did not RTFA) is that some languages are cheaper, IIRC even has a super-caveman mode where it responds in Chinese/Japanese because that costs fewer tokens.

In the end LLM are just predicting what is statistically the most likely follow-up for a sentence, according to what they've been trained on. That's how they work.

For this they cut sentences into bits called tokens. The text they are working on is a sequence of tokens, and they predict the next one.

A token could be a whole word (generally if it's short and usual), often it's a syllable, and if you're trying to use uncommon words/names they might even cut them down letter-by-letter in part to try and fit them into something they can swallow.

Of course tokens would be very language dependent, and some languages just use more characters/longer words and sentences than others on average.

I'm going to add to the previous responses that a token is also the billing unit used.

If you hire a lawyer, they charge a rate based on their predetermined "billable hours" arrangement. They beak down their jobs into small time segments, often much smaller than an actual hour. Similar to how a mechanic might use a predetermined labor cost based on a time estimate chart; swapping a battery takes X time, but rebuilding an engine takes Y time, with the actual number of hours and minutes not being useful in terms of billing because neither lawyers or mechanics can precisely track exactly how much of the time they're digging working on your specific job is being spent "hands on" vs the time necessary to do all the little things that come up to interrupt the work. So they bill by the abstracted time a job takes when stripped of real world necessity.

Tokens are, as others said, chunks of data. When you enter a prompt, and get a response, those chunks get tallied up and billed for. From what I've read (and I'm not in the field, not even as s hobbyist/amateur; I just wanted to know exactly what you asked) the various companies bill by large chunks, usually per million tokens, or a similar chunks.

As an example, a given word might only need three tokens to process as input, and another three as output. But, because the cost of doing that work is more intensive to output (generate) a response, output costs more per unit. Now, I'm not saying the capitalist hellscape reality of that is a good thing, I'm just saying that's the reasoning of the capitalist twats that run the companies. Anywhere other than lemmy, I might not need that disclaimer lol.

Now, why they chose token as the jargon, I've never looked to see if I could find an answer, but that's just me wondering just enough to type it, but not enough to go looking ;)

Tf is a token?

If you plugged this question into an LLM here's how it would be broken into input tokens:

BOS (Beginning of Sequence) | Tf | is | a | token | ? | EOS (End of Sequence)

So this sentence would be 7 input tokens into an LLM. The sentence before this one, which is my response, would be 15 output tokens. The big LLM providers charge you per token used (usually the 1000 tokens used). Some LLM providers charge one price for input tokens and another for output tokens.

Here's how that translates to actual dollars to you. Here's an price list from Oracle Cloud, a large company providing LLM service. Lets take the Google Gemini 2.5 Pro LLM model hosted at Oracle Cloud:

• Oracle Cloud Infrastructure Generative AI - Google - Gemini 2.5 Pro - Input Tokens - Text, Image, Audio, and Video less than 200K input tokens = $1.25/1,000,000 tokens

• Oracle Cloud Infrastructure Generative AI - Google - Gemini 2.5 Pro - Output Tokens - Text Output less than 200K input tokens = $10/1,000,000 tokens

So your input question would have cost you $0.0000175 and my single sentence response to you would have cost you $0.00015.

Some LLMs (like those discussed in the article) separate each letter as a separate token for processing as input. In that case your same question in one of those LLMs would be 13 tokens instead of 7.

So the title should read that the AI companies are gouging some of their customers because they can.

But that's not what the underlying study shows. It found that LLMs (including the ones you can self-host, not just the premium models) require more tokens to answer the same prompt in foreign languages versus in English. This is almost certainly a consequence of English being over-represented in the training data (and to a lesser extent, tokenizers that were designed with English in mind). It's not some intentional and nefarious price gouge.

Don't get me wrong, I know for a fact that the frontier labs are engaged in all sorts of nefarious anti-consumer shit — but this ain't that.