I'm going to add to the previous responses that a token is also the billing unit used.

If you hire a lawyer, they charge a rate based on their predetermined "billable hours" arrangement. They beak down their jobs into small time segments, often much smaller than an actual hour. Similar to how a mechanic might use a predetermined labor cost based on a time estimate chart; swapping a battery takes X time, but rebuilding an engine takes Y time, with the actual number of hours and minutes not being useful in terms of billing because neither lawyers or mechanics can precisely track exactly how much of the time they're digging working on your specific job is being spent "hands on" vs the time necessary to do all the little things that come up to interrupt the work. So they bill by the abstracted time a job takes when stripped of real world necessity.

Tokens are, as others said, chunks of data. When you enter a prompt, and get a response, those chunks get tallied up and billed for. From what I've read (and I'm not in the field, not even as s hobbyist/amateur; I just wanted to know exactly what you asked) the various companies bill by large chunks, usually per million tokens, or a similar chunks.

As an example, a given word might only need three tokens to process as input, and another three as output. But, because the cost of doing that work is more intensive to output (generate) a response, output costs more per unit. Now, I'm not saying the capitalist hellscape reality of that is a good thing, I'm just saying that's the reasoning of the capitalist twats that run the companies. Anywhere other than lemmy, I might not need that disclaimer lol.

Now, why they chose token as the jargon, I've never looked to see if I could find an answer, but that's just me wondering just enough to type it, but not enough to go looking ;)