The Unit of AI Commerce You’ve Never Seen
Every interaction with a commercial AI language model is priced in tokens — the unit that determines how much an API call costs and influences how the model processes language. Most users interact with AI through interfaces that abstract away the token layer entirely: you type a question, you get an answer, and you pay a monthly subscription without knowing or caring how many tokens were involved. The token concept becomes practically important when you’re building applications on AI APIs (where you’re paying per token), when you’re hitting context length limits, and when you’re trying to understand why a specific prompt produces better or worse results.
Understanding tokens doesn’t require understanding the mathematical details of tokenisation. What’s useful is the intuition: what a token roughly corresponds to, what the limits mean in practice, and how token counting affects the cost and performance of AI-powered applications.
What a Token Actually Is
A token is approximately 3-4 characters of English text, or roughly 0.75 words. Common words are often a single token (‘the’, ‘is’, ‘and’, ‘computer’); longer or less common words may be multiple tokens (‘tokenisation’ might be two tokens: ‘token’ and ‘isation’). Punctuation and spaces are separate tokens or parts of tokens depending on their position. Numbers, code, and non-English text have different tokenisation patterns — code tends to be more tokens per character than prose because the character set includes many uncommon sequences; Chinese characters might each be a single token despite representing a full word.
The practical approximation: 1,000 tokens is approximately 750 words of English text. A long essay of 3,000 words would be approximately 4,000 tokens. This approximation is imprecise because tokenisation varies by content type, but it’s useful for rough cost estimates and context window planning.
Context Windows: The Memory That Tokens Fill
The context window is the total number of tokens a model can process in a single interaction — the combined length of the input (your message, any system instructions, conversation history) and the output (the model’s response). When the context window is full, the model can’t process additional information: either earlier parts of the conversation are dropped, or the API returns an error for exceeding the limit.
Context window sizes have grown enormously: GPT-4’s original 8K context has expanded through successive versions to 128K and beyond; Claude 3 Opus and Sonnet support 200K tokens; Gemini 1.5 Pro has demonstrated million-token context in benchmarks. These large context windows enable previously impossible tasks: analysing an entire book in a single prompt, including the full content of a lengthy codebase in the context, or maintaining a very long conversation without losing context from earlier in the thread. The cost scales with context size, making large context use appropriate for tasks that genuinely require it rather than as a default.
How Tokens Affect API Costs
Commercial AI APIs price based on token count, with separate rates for input tokens (what you send) and output tokens (what the model generates). Output tokens are typically more expensive than input tokens because they require more computation to generate. At OpenAI’s gpt-4o pricing as a reference point, input tokens cost a fraction of the output token rate, creating an incentive to optimise prompts and request concise outputs when cost is a consideration.
For application developers, token cost optimisation produces meaningful savings at scale: choosing a smaller, cheaper model for tasks that don’t require frontier capability, caching repeated context components rather than sending them in every call, requesting shorter outputs when length isn’t necessary, and batching queries where the API supports it. These optimisations can reduce API costs by 50-80% for applications with significant volume without reducing output quality for appropriate use cases.
Practical Token Awareness for Non-Developers
For users interacting with AI through consumer interfaces rather than APIs, token awareness produces specific practical improvements. Context window limits explain why very long conversations sometimes produce responses that seem to have ‘forgotten’ earlier parts of the conversation — the earlier content has scrolled out of the context window. Starting a new conversation rather than continuing a very long one, or summarising the important context explicitly in the new conversation, addresses this limitation.
Prompt length awareness: extremely long prompts (pasting entire books, including large amounts of reference material) may approach context limits or produce slower responses. Being more selective about what context is provided — including the specific sections relevant to the question rather than entire documents — produces faster responses and, often, better-targeted ones, because the model isn’t competing for attention against irrelevant included material.