Understanding Tokens: What They Are and Why They're Important

Tokens are Parts of Words

Most people think AI, like ChatGPT, reads words. It doesn’t.

It reads tokens — invisible chunks of text that power every interaction.

When you type something like:

Hello, world!

The model doesn’t see two words. It sees four tokens:

• Hello → 1 token

• , → 1 token

• world (note the space) → 1 token

• ! → 1 token

That simple greeting is 4 tokens, not 2 words. Code fragments break into even more tokens because punctuation, brackets, and symbols all get split up. (What is and isn’t a token and what becomes one actually depends on the model, so our example isn’t exact.)

Tokens Aren’t Expensive. Processing them is.

When you send your tokens to get processed, each one must be run through billions of math operations on very expensive GPUs every single time. That’s where the cost comes from — the hardware needs:

• A lot of power to run

• Datacenter space to store and run them, and datacenters are expensive

• Constant, special cooling, because it generates huge amounts of heat

• Staff to maintain it and keep it secure and to keep it running optimally

More tokens → more GPU time → higher costs.

Fewer tokens → less GPU time → lower costs.

Right now, you probably don’t see the meter running. You pay a flat subscription; someone else covers the token bill.

Under the hood, tokens are the biggest driver of compute costs at every AI company.

Tokens are the Foundation for Everything Else

• Context windows, or how much a model can “see” at one time, are measured in tokens.

• API pricing is per million tokens (API access is when companies or developers access an AI model to provide their own service, like a chatbot on a website, or just for internal use).

• Memory, efficiency, and much of prompt engineering are all about how tokens are used.

If you want to get the basics of how AI works, you have to start with tokens. Tokens are the building blocks for AI, and a big part of why it sometimes costs so much.

Learn more about how Knitli is tackling the hidden economics of AI at the source, visit us at knitli.com and subscribe to our waitlist for updates!