If you use an LLM through an API, you will meet three terms quickly: tokens, context window and pricing. Understanding them helps you predict and control costs.
What is a token?
A token is a small chunk of text, often part of a word. LLM providers usually charge based on the number of tokens you send in and the number the model generates in reply.
What is a context window?
The context window is the maximum amount of text a model can consider in one request, including your instructions, any documents you include, the conversation so far and the answer. Larger windows allow longer documents but generally cost more to use.
Practical ways to control cost
- Send only the information the task needs, not entire documents
- Use retrieval to pull relevant passages rather than pasting everything
- Choose a smaller, faster model for simple tasks and a stronger one for hard tasks
- Set limits on response length
- Cache repeated instructions and answers where possible
- Monitor usage per feature so surprises are caught early
Estimate before you build
Take a realistic sample, count the text volume per request, multiply by expected daily use and compare with the provider’s current price list. Prices change, so always check the latest figures.
We can help you design AI features that stay within budget. See our AI & Automation service or get in touch.