How to Reduce LLM Token Usage

AI models read and write in tokens: pieces of words, numbers, and punctuation. Tokens decide what a request costs, how much fits in the context window, and how long a reply takes. The same information can take very different numbers of tokens depending on how you send it.

The token counts on this page are exact, measured with OpenAI's o200k_base tokenizer (used by GPT-4o and newer). Claude and Gemini use their own tokenizers, so their counts differ, but the same changes save tokens there too.

1. Measure before you change anything

Paste your prompt into the AI Token Counter to see where the tokens go. Usually one part dominates: a pasted document, a block of JSON, or a long chat history. Start there.

2. Send records as a table, not JSON

JSON repeats every field name in every record, and pretty-printing adds indentation and line breaks. Here is one list of 20 orders, each with six fields, sent four ways:

CSV names each field once, in the header row:

id,customer,status,items,total,currency
ORD-10421,Ada Lovelace,shipped,1,19.5,USD
ORD-10422,Grace Hopper,processing,2,26.75,USD
ORD-10423,Alan Turing,delivered,3,34,USD
…

Models read tables well, especially with a one-line note above the data, such as "The data below is CSV with a header row; each row is one order." Two things to watch:

• CSV stores text, so the model can't tell the number 1 from the text "1". That rarely matters for reading, but if types matter, use columnar JSON (field names once, then rows of values).
• Nested objects need flattening into columns such as address.city.

JSON to LLM Table does the conversion, flattening, and counting for you.

3. Remove the padding

Text pasted from documents, web pages, and chats collects things a model doesn't need: runs of spaces, stacks of blank lines, a line pasted twice, long rows of dashes, and indented JSON. This short prompt is 93 tokens:

You are a careful   support assistant.



Use the supplied context to answer the question.
Use the supplied context to answer the question.
------------------------------------------------------------

Context:
```json
{
  "order": {
    "id": "ORD-10421",
    "status": "shipped",
    "carrier": "UPS",
    "eta": "2026-10-16"
  }
}
```

============================================================
Question:   Where is my order?

With the padding removed, it's 61 tokens (34% fewer), with the same words in the same order:

You are a careful support assistant.

Use the supplied context to answer the question.
---

Context:
```json
{"order":{"id":"ORD-10421","status":"shipped","carrier":"UPS","eta":"2026-10-16"}}
```

===
Question: Where is my order?

The Prompt Compressor applies these rules and leaves code blocks exactly as they are, because spacing can matter in code.

4. Only send what the task needs

• Drop unused fields. If the question is about order status, internal IDs, timestamps, and audit fields are just cost.
• Trim logs and documents to the relevant section instead of pasting the whole file.
• Shorten long conversations. Replace old turns with a short summary, and keep only the recent ones in full.

5. Keep the repeated part first, and ask for shorter replies

Most major AI providers offer prompt caching: when many requests start with the same text, that repeated beginning is processed faster and billed at a lower rate. It only works if the unchanging part (instructions, examples, reference material) comes first and the changing part (the user's question) comes last. Check your provider's documentation for the details.

Output tokens are usually priced several times higher than input tokens, so replies matter too. Ask for the format you need ("answer in one sentence", "return only these three fields"), and set a maximum output length.

Checklist

• Count tokens first, and fix the biggest part.
• Send lists of records as CSV or a Markdown table, not pretty JSON.
• Remove spacing, repeated lines, and decorative separators.
• Leave out fields, logs, and history the task doesn't need.
• Put stable instructions first so they can be cached, and ask for short replies.