How to Reduce AI Coding Assistant Costs: 10 Token Efficiency Strategies

AI coding assistants can navigate repositories, modify several files, run tests, and iterate on failures. That capability comes with a less visible resource: the model context consumed at every step. The longer a session runs, the more files, command outputs, tool definitions, and earlier messages may be sent back to the model. As a result, a seemingly small coding task can become expensive, slow, and increasingly difficult for the assistant to reason about.

Token efficiency is therefore not simply about writing shorter prompts. It is the discipline of giving an assistant the smallest sufficient context for the current decision, while preserving the evidence needed to produce a correct and verifiable result.

The stakes are no longer theoretical. In 2026, Uber exhausted its entire annual AI budget by April - four months in - after Claude Code adoption spread to roughly 5,000 engineers faster than finance models had anticipated; monthly cost per engineer averaged $150–250, with power users spiking to $500–2,000 (Janakiram, 2026). Amazon disclosed a comparable pattern: an internal Claude Sonnet deployment for matching product listings cost $1.8 million, 860% over its original budget, with the overrun going unnoticed for nearly five months, while two other internal projects separately exceeded budget by $541,000 and $134,000 (Udinmwen, 2026). Both cases share the same root cause: token-metered, agentic coding tools do not behave like flat-rate software line items, so unmanaged context growth can turn routine engineering work into a runaway bill before anyone notices.

This article presents practical measures for developers and platform teams, based primarily on the official guidance from GitHub Copilot and Claude Code. It also evaluates a growing ecosystem of tools for repository indexing, selective retrieval, output compression, and compact data representation.

What consumes tokens in an AI coding workflow?

A model request can contain much more than the developer’s latest instruction. Depending on the assistant, it may include conversation history, selected source files, repository instructions, tool descriptions, terminal output, diffs, test reports, retrieved documentation, and the assistant’s previous plans or results. Generated code and explanations add output tokens as well.

Figure 1. Cost is price per token multiplied by token volume. The left side shows what moves the price - model tier, input versus output, cached versus uncached - with rough multipliers; the right side lists what fills the context on every request. The numbered badges point to the practice in this article that addresses each driver.

Coding workloads are especially context-intensive because agents often repeat a cycle of planning, searching, reading, editing, testing, and correcting. Each step can add new material or resend earlier context. Large files, verbose logs, broad repository scans, and unsuccessful attempts can quickly dominate the useful information in the session.

The goal is not to minimize tokens at any cost. Too little context can cause incorrect edits, missed dependencies, and additional correction rounds. The relevant optimization target is tokens per successfully completed, validated task.

Ten practices that deliver the greatest impact

The practices below are ordered by leverage. The first few shape every later step of an agent session, so they pay off on almost every task; the last few are situational levers worth applying once the fundamentals are in place.

1. Write precise, scoped prompts

A vague request encourages broad searching and repeated interpretation. A better prompt defines the objective, boundaries, acceptance criteria, and relevant starting points.

Avoid- vague and unscoped: 

Prefer- objective, boundaries, and acceptance criteria: 

The same contrast applies when fixing a failing test.

Avoid- no starting point or scope: 

Prefer - names the failure, scope, and verification step: 

This prompt is longer, but it reduces total consumption because it prevents unnecessary exploration and correction - and by reaching a correct result in fewer round trips, it also shortens the time to a working change. Include only information that changes the assistant’s decisions. Prefer file paths, symbols, failing tests, diffs, and exact error messages over a long narrative.

2. Match the model- and its reasoning effort- to the task

Using the most capable reasoning model for every request is rarely cost-efficient. GitHub recommends reasoning models for architecture, complex debugging, and system design; mid-tier models for executing a clear plan; and lighter models for routine refactoring, formatting, or documentation. GitHub Copilot’s automatic model selection can also route requests according to task requirements and avoid switching models in the middle of a cacheable task.

A practical team policy could be:

• Lightweight model: formatting, renaming, boilerplate, documentation, and simple tests
• General-purpose model: scoped feature implementation, familiar refactoring, and routine debugging
• Reasoning model: architecture decisions, unfamiliar failure modes, cross-system changes, and security-sensitive analysis

Model size is only one lever, though. The reasoning-effort setting can matter just as much: it controls how much inference-time computation a model spends before responding. Coding tools often choose a middle setting by default, while published

benchmarks may use high, xhigh, or max reasoning, so a model that looks weak at medium effort can become substantially more useful when allowed to reason longer - often at a lower inference cost than switching to a larger model. The figures and index scores discussed here are based on the Artificial Analysis Coding Agents leaderboard.

Figure 2. Coding Agent Index against cost per task for the GPT-5.6 Luna, Terra, and Sol families, each at five reasoning-effort levels; the cost axis is logarithmic. Luna at max reasoning lands within two index points of Sol at medium for roughly a tenth of the cost. Source: Artificial Analysis Coding Agents leaderboard; the values match the table below. 

In the GPT-5.6 family, for example, increasing the effort of the lightweight Luna model can deliver a near-frontier result at much lower cost than moving directly to the larger Sol model at its default medium setting. GPT-5.6 Luna at max reasoning reaches an index score of 59 at an estimated $0.31 per task in the displayed benchmark - only two index points below GPT-5.6 Sol at medium reasoning ($2.99), while reducing the estimated cost per task by about 90%. Luna at x-high effort costs $0.25, about 92% less than Sol at medium, although its index score is six points lower. For many tasks, the smaller model at higher effort is therefore the more economical choice when near-frontier performance is sufficient. The saving comes at the cost of wall-clock time, though: in the same benchmark, average agent time per task rises from about 5.2 minutes for Sol at medium reasoning to about 8 minutes for Luna at max reasoning. Where a developer is blocked waiting on the result, that extra 2.8 minutes per task can matter more than the lower cost.

Practical recommendation: Before upgrading to a larger model, test the lightweight model at high or max reasoning on representative tasks. Compare not only benchmark scores, but also cost per successfully completed task, latency, and the number of correction rounds. Set the reasoning level before the session begins so that prompt caching is not disrupted.

Interpretation note: The percentages are calculated from the displayed benchmark values and rounded to the nearest whole percent. Benchmark results depend on the agent harness, task set, reasoning setting, and pricing assumptions; validate the choice with your own repository and workload before defining a team-wide default. 

3. Separate planning from execution

For complex work, resist the temptation to research, plan, and implement in one continuous session. GitHub recommends breaking the work into distinct phases: use a capable reasoning model to explore the codebase and produce a concise, reviewable plan, let a developer approve or adjust it, and then execute that plan with a less expensive model - ideally in a fresh session that starts from the approved plan rather than the full planning history.

The core reason to split these phases is that research and planning are context-intensive by nature. Exploring a repository pulls in files, search results, discarded options, and dead ends, and every one of those tokens is resent on each subsequent turn if implementation continues in the same conversation. Starting execution in a new session discards that accumulated context and reintroduces only the finished plan, so the implementation model works from the smallest sufficient context instead of a transcript of how the plan was reached.

A leaner context also produces more faithful execution. When the planning exploration no longer competes for the model's attention, the agent is far less likely to revisit

superseded reasoning, reopen an approach the plan already rejected, or drift from the agreed scope; each phase operates with only what it needs to make the next correct decision. This pattern therefore delivers three compounding benefits:

1. Expensive reasoning is used only where it adds value, while execution runs on a cheaper model.

2. The execution agent receives a smaller, purpose-built context package and follows the plan more precisely.

3. Human review occurs before broad repository changes consume additional tokens.

Starting a fresh session for execution does discard the planning session's prompt cache, but this is rarely a real cost: moving from a reasoning model to a cheaper execution model already invalidates that cache (see practice 6), and the small, stable plan quickly builds a new cache that is reused across the many turns of implementation.

4. Use deterministic workflows wherever possible

AI agents are probabilistic: the same instruction can produce different plans and outcomes, and small mistakes can compound across a multi-step workflow. Therefore, do not ask the model to reason through a task that can be executed reliably by an existing deterministic tool. Repetitive operations such as running a test suite, applying a formatter, linting code, scanning dependencies, fetching AWS logs, or collecting diagnostic information should be implemented once as scripts or standard commands and then invoked by the agent.

Package these scripts as reusable skills with a narrow interface, documented inputs, and predictable outputs. The agent then only needs to select and call the appropriate capability instead of repeatedly generating commands through trial and error. Anthropic’s web-application testing skill illustrates this pattern: bundled helper scripts are treated as black boxes and called directly, avoiding unnecessary source-code ingestion while providing a repeatable way to manage servers and execute tests.

• Unit and integration tests: verify that the generated changes satisfy expected behavior and expose regressions before the agent builds on an incorrect result.
• Formatters and linters: enforce structure and consistency automatically instead of spending model turns on style corrections.
• Security and policy scans: detect secrets, vulnerable dependencies, unsafe patterns, and prohibited changes with explicit pass/fail signals.
• Operational scripts: retrieve logs, start local services, reproduce failures, or gather diagnostics in a stable, pre-tested form.

Run these checks locally for fast feedback, and again in CI/CD as the gate that decides whether the change ships. That gives a tight loop: the agent makes a change, a deterministic check judges it, and the agent gets a pass/fail result before moving on. A test or scan costs a few seconds. Skipping it and letting the agent guess costs more: one bad guess can trigger several more edits to patch the last mistake, each adding logs and diffs to the context. Catching the problem at the source keeps the task shorter, cheaper, and easier to trust.

Practical rule: use the model for judgment, ambiguity, and adaptation; use scripts, tests, hooks, and CI/CD for repeatable execution and enforcement. If a workflow can return an unambiguous exit code or structured result, prefer that signal over asking the model to decide whether its own work is correct.

5. Keep long conversations lean: compact or restart at the right time

Long conversations accumulate context debt: failed approaches, superseded requirements, repeated logs, and assumptions that no longer apply. Claude’s cost guidance highlights context management, model selection, extended-thinking settings, and preprocessing hooks as key levers for reducing usage. Claude Code’s /usage view can attribute recent usage to areas such as skills, subagents, plugins, and MCP servers, making context growth more observable.

The growth is easier to see in a worked example. Here, a "turn" means one carried-forward interaction in the same conversation without compaction1 or a restart; a genuinely fresh conversation would not retain the entire transcript. The figures are illustrative estimates and not provider telemetry. Each estimate represents the full working context sent for that turn, including retained history, instructions, selected files, tool output, diffs, and test results.

Figure 3. An illustrative eight-turn context-growth scenario. Context reaches 175k tokens before Turn 7 compacts the conversation, then falls to 45k in Turn 8. Rough totals use the supplied GPT-5.6 Sol price snapshot: $5 per 1M input tokens and $30 per 1M output tokens, with a fixed 2k output estimate per turn. Cached input and cache-write charges are excluded; actual output varies.

Added is the initial context for Turn 1, the newly introduced context through Turn 7, and the context reduction shown after compaction in Turn 8. Rough cost* combines uncached input at $5 per 1M tokens with a fixed 2k output estimate at $30 per 1M tokens; it excludes cached input and cache-write charges. Cumulative input sums the full working-context estimate sent on every turn, not the visible prompt text alone. The compacted handover is intentionally much smaller than the pre-compaction transcript, so Turn 8 starts from 45k rather than resending 175k.

Compact after a meaningful milestone, such as after diagnosis and before implementation. Start a fresh conversation when the goal changes. Carry forward a structured handover containing:

• Objective and scope
• Decisions already made
• Files and symbols involved
• Current test status
• Remaining tasks and risks

When compacting, add a short instruction that tells the assistant which information must survive the summary. For example, /compact Focus on code samples and API usage prioritizes those details instead of relying on a generic summary. Adapt the instruction to the current task-for example, preserve architectural decisions, failing test output, modified files, unresolved risks, or exact API contracts. Custom compaction instructions make the reduced context more useful and decrease the likelihood that missing details must be rediscovered later. Compaction only works after the session has accumulated conversation history; in a fresh session, Claude Code reports Not enough messages to compact.

6. Preserve the prompt cache across a coding session

Prompt caching allows an AI model to reuse parts of the context it has already processed, including system instructions, file contents, conversation history, and tool definitions. This is particularly valuable in agentic coding workflows because the same large context is often submitted repeatedly over many turns. According to GitHub, cached tokens are typically billed at 10% of the normal input-token price, although the exact rate depends on the selected model.

Understanding how the cache actually decides what to reuse helps explain the practical rules below. Anthropic's Claude Code guide describes the mechanism as prefix matching: the model re-processes the full request on every turn, and the cache matches the start, or prefix of each request against content it recently processed. The match is exact, so a change anywhere in the prefix recomputes everything after it; there is no per-file or per-segment caching. To take advantage of this, coding assistants build each request with the most stable content first - system prompt and tool definitions, then project context such as AGENTS.md, then the growing conversation - so a change lower down leaves everything above it cached. A change in the conversation layer leaves the earlier layers cached, whereas a change to the system prompt or tool set invalidates everything behind it. On a normal turn the cached prefix is effectively the entire previous request, and only the latest exchange is new-exactly the assumption used in the estimate below.

The cost effect is visible across all eight turns of the preceding example. The figure below reuses the same working-context estimates, but treats the entire previous turn as cached from Turn 2 through Turn 7. After compaction, Turn 8 carries 8k from the compacted handover as cached input, while its remaining 37k is newly introduced. Hatched portions therefore show input that can use the discounted cache-read rate; solid portions are newly introduced input charged at the normal rate.

Figure 4.  An illustrative eight-turn caching scenario. Turn 1 establishes the cache. Turns 2-7 treat the entire previous turn as cached at 10% of the $5 per million input-token price, while new input remains full price. Turn 8 follows compaction with 8k of its handover cached at 10% and 37k of new context charged at full input price. Every turn includes the same fixed 2 thousand output-token estimate. This is a full-request estimate, not provider telemetry; cache eligibility, cache-write charges, and actual output vary by model and provider. 

*Request with cache policy includes the previous turn's cached input at 10% of the normal rate, newly added input at $5 per million tokens, and a fixed 2 thousand output-token estimate at $30 per million tokens. For Turn 8, compaction retains 8k of the handover as cached input, so its remaining 37k is charged at the normal input rate. The uncached equivalent sends the full current working context at the normal input rate with the same output estimate. Values are rounded for display; Turn 8's small cache-read charge is shown to three decimal places so the nonzero amount remains visible; totals use unrounded calculations, and cache-write charges are excluded.* 

Under these assumptions, caching reduces the eight-turn estimate from about $4.03 to $1.79, saving $2.24 or 56% overall. The per-turn saving grows from about 38% in Turn 2 to 82% in Turn 7 because the cached portion grows from 20 thousand to 170 thousand tokens while only the newly added input pays the full input rate. Turn 8 retains a smaller cache benefit of about 13% because 8 thousand tokens of the compacted handover remain cached, reducing its request estimate from $0.29 to $0.25. Caching does not make the whole request 90% cheaper: output, new input, and post-compaction context still cost money, but repeated context becomes much less expensive while the cache remains valid.

To benefit from caching, keep the session configuration stable. Select the model, reasoning level, context size, and enabled tools before starting the task, then avoid changing them while the assistant is working through a coherent set of steps. The Claude Code guide sums up the habit well: pick your model and effort level at the top of a session, then save compaction for natural breaks between tasks. The fewer changes you make mid-task, the higher your cache hit rate.

It helps to know which mid-task actions break the prefix and which are safe. The following actions invalidate the cache and force an expensive, uncached turn while it rebuilds:

• Switching models: A model cannot reuse another model's cache-the model is part of the cache key, not the prompt text, so an identical context is still recomputed. Select a model at the beginning of the task, or use Copilot auto model selection, and keep it for the session.
• Changing reasoning or tool settings: Changing the reasoning-effort level or context size recomputes the whole request, and adding, removing, connecting, or disconnecting tools, MCP servers, or plugins that load tool definitions changes the system-prompt layer that sits at the front of the prefix. Configure these before starting and avoid changing them during the session.
• Upgrading the assistant mid-session: A new version typically changes the system prompt or tool definitions, so the first turn afterward rebuilds the cache from the top.

By contrast, several common actions keep the cache warm because they only append to the end of the conversation: invoking skills and commands, generating a terminal-only recap, switching permission mode, and editing repository files (the assistant appends a change notice and re-reads on demand rather than rewriting history). When you want to abandon a dead-end path, prefer rewinding to an earlier turn over compacting: rewinding truncates back to a prefix that is already cached, whereas compaction builds a new one.

Time is a factor too. Cached prefixes expire after a period of inactivity, and every cache hit resets the timer. GitHub states that caches expire after 24 hours of inactivity for OpenAI models and after one hour for most other models; the Claude Code guide describes a five-minute default that a Claude subscription automatically extends to one hour. After a longer break, the first turn back re-processes the full history uncached, so start a new session or run /compact so that the context is rebuilt from a concise summary rather than the complete conversation history.

Finally, verify that caching is actually working rather than assuming it. Coding assistants report a cache-read token count (billed at roughly 10% of the input rate) alongside a cache-write count; a high read-to-write ratio means the prefix is stable, while write counts that stay high turn after turn signal that something in the prefix keeps changing.

7. Keep AGENTS.md short, curated, and human-owned

Persistent instruction files and tool definitions consume context before the assistant begins the actual task. AGENTS.md and CLAUDE.md are essentially the same artifact: an instruction file that the assistant loads automatically, whether placed at the user, organization, repository, or subdirectory level. AGENTS.md is the tool-agnostic convention adopted across many assistants, while Claude Code reads a file named CLAUDE.md; the rest of this section uses AGENTS.md to mean any such file. These files are valuable when they capture non-obvious coding conventions, canonical commands, architectural boundaries, and quality gates. However, every globally loaded instruction competes with the code and evidence needed for the current decision. The file loads automatically at the start of every session, so any detailed procedure it contains stays in the base context even on tasks where that procedure is irrelevant.

Do not generate AGENTS.md automatically and accept the result as repository policy. Gloaguen et al. found that LLM-generated repository context files did not improve task completion and that context files increased inference cost by more than 20% on average. The agents generally followed the added instructions, but the extra requirements encouraged broader file exploration, additional testing, and more tool calls; repository overviews were particularly unhelpful. Automatically generated guidance can therefore add plausible-looking context that makes routine tasks more expensive or harder without contributing information the agent could not discover itself.

Instead, treat the file as a small, human-owned configuration artifact. Maintainers should add only minimal, verified requirements that are both non-standard and broadly applicable. For example, the canonical build and test commands, unusual package-manager choices, architectural boundaries that must not be crossed, and mandatory validation or security rules. Omit generated directory trees, repository summaries, discoverable technology descriptions, duplicated README content, and style rules already enforced by formatters or linters. Review every proposed line in a pull request, measure whether it improves representative tasks, and remove instructions that do not demonstrably influence correct behavior. Keep always-on instruction files limited to guidance that applies broadly and consistently. Move task-specific procedures, such as pull-request reviews, release processes, database migrations, or incident response into Skills that are loaded only when relevant. For example, replace a 100-line migration procedure in AGENTS.md with a dedicated database-migration skill that can be invoked when needed.

Apply the same principle to tools. Large tool collections-for example, a full MCP server with many available operations-add tool descriptions to the context on every request. Where the workflow permits, enable only the MCP servers, plugins, and toolsets required for the current task. A focused tool configuration reduces baseline context, narrows the assistant’s choices, and lowers the risk of irrelevant tool calls.

Both moves follow the same principle of progressive disclosure: keep the always-on baseline small, and reveal detailed procedures and tool definitions only at the moment a task requires them.

As a practical rule of thumb, keep AGENTS.md below roughly 200 lines and review persistent context like production code. Remove duplicated guidance, outdatedexceptions, unused tools, and prose that does not influence behavior. Before starting a task, ask two questions: “Must the assistant know this on every request?” and “Must this tool be available for this task?” If not, load it on demand or leave it disabled.

8. Use English as your standard language

Natural-language text is converted into tokens before it is processed by a model, and equivalent instructions do not necessarily require the same number of tokens in every language. Because many model tokenizers and training corpora are strongly optimized around English, English often represents common words and technical expressions more compactly than languages such as German or Japanese. For coding assistants, this makes English a sensible default for persistent instructions that are sent repeatedly-especially AGENTS.md, skill files, repository conventions, and reusable prompt templates.

The potential saving is easy to illustrate with the same Git instruction translated into four languages. Measured with the OpenAI tokenizer for GPT-5.x and O-series models, the same instruction grows from its English baseline to about 76% more tokens in German and more than twice as many in Japanese, as the table below shows. The exact ratios are model- and tokenizer-dependent, but repeated differences matter when the same instruction files are included in every request.

Example prompt used for the comparison: “Review my current Git branch, identify all commits not yet merged into main, propose an optimal interactive rebase strategy to create a clean commit history, and generate the exact Git commands required.”

In practice, standardize reusable technical instructions in English when the team can review them reliably, but do not force English into every interaction. Most coding-assistant context is often source code, diffs, tool definitions, and command output, so a one-off prompt may account for only a small share of total cost. A clear, precise prompt in the developer’s strongest language is preferable to a vague or error-prone English prompt that causes additional searches, corrections, or failed attempts.

Language	Token count	Additional tokens vs. English	Relative token use
English		Baseline	1.00×
Chinese		+17 (+59%)	1.59×
German		+22 (+76%)	1.76×
Japanese		+33 (+114%)	2.14×

The same principle applies directly inside source files. Docstrings, comments, and inline explanations are often read repeatedly by coding assistants when they inspect functions, build summaries, generate tests, or revisit the same file after a failed attempt. A German docstring may be perfectly understandable to a German-speaking team, but it can add avoidable token overhead and make the surrounding technical context less uniform for the assistant.

Avoid this pattern in shared repository code: 

Prefer the English version: 

This looks harmless, but it becomes expensive when the same file is loaded again and again during an agentic workflow. The assistant may read this function while planning a change, inspect it again while editing related tests, revisit it after a failed test run, and include it in later summaries or diffs. Each repetition resends the non-English natural-language explanation, increasing context size without adding information that could not be expressed more compactly and consistently in English.

English docstrings also create a cleaner shared surface for mixed teams, open-source conventions, external libraries, and coding assistants trained heavily on English technical material. The recommendation is not to ban local-language communication, but to keep reusable technical context-docstrings, comments, repository instructions, examples, and prompt templates-in English when those files are likely to be consumed repeatedly by AI tools.

This recommendation is a default, not a rule, and there are good reasons to override it. The underlying goal is a single, consistent language for the repeated technical context in a repository or session - English is simply the most token-efficient choice for most tokenizers, not an end in itself. Where a stronger reason points elsewhere, follow it: a team that reads English poorly will produce and review better code in its own language; software built for a specific market may need local-language identifiers and documentation; and some clients or regulated domains explicitly require code and docstrings in a particular language. In those cases the comprehension and correctness gains outweigh a modest token overhead. What rarely makes sense is a third language that no one on the team commands well - configuring the harness, instructions, and prompts in a language you do not speak fluently, while English would serve you better, adds cost without any offsetting benefit.

9. Don't use C++ and JSON; at least not only for token efficiency

The heading is intentionally not entirely serious: nobody should rewrite a suitable C++ system merely to save prompt tokens. Language selection must still be driven by runtime requirements, ecosystem, safety, maintainability, team expertise, and compatibility. Nevertheless, programming languages encode equivalent logic with different amounts of syntax, boilerplate, and type information, so a repository’s primary language can affect how much source code fits into an agent’s context window. Martin Alderson’s exploratory comparison of like-for-like Rosetta Code solutions found a substantial spread between programming languages, with concise dynamic and functional languages generally using fewer tokens than more verbose low-level languages. The analysis is useful as a signal, not a language-selection benchmark: the author explicitly notes limitations and biases in the dataset, and token count says nothing by itself about correctness, performance, security, generated-code quality, or the number of iterations an agent needs. Treat language efficiency as a secondary factor when choosing a new technology, not as a reason to abandon an established C++ codebase.

The following table reproduces the Token Calculator ranking based on average tokens for Rosetta-style tasks, with the original verdict column omitted.

Source: Token Calculator, “Most Token-Efficient Languages for LLMs, Ranked & Priced,” based on Rosetta-style programming tasks and tokenizer measurements. The original verdict column has been omitted here. 

Token efficiency is also not the same as overall efficiency. A language or representation that is compact for an LLM may still be inefficient once the generated or maintained code is executed. For AI workloads in particular, Marini et al. show that programming-language choice can materially affect energy consumption: in their controlled GREENS 2025 experiment across C++, Java, Python, MATLAB, and R, compiled and semi-compiled languages generally consumed less energy than interpreted languages, which in some cases required up to 54 times more energy. They also emphasize that the most energy-efficient choice depends on the algorithm, training versus inference phase, implementation, and development trade-offs. Therefore, language and format decisions for AI-assisted software should consider token cost, maintainability, ecosystem fit, runtime performance, and energy efficiency together. For structured payloads sent to a model, the format is easier to change. Token-Oriented Object Notation (TOON) is a lossless representation of the JSON data model designed for LLM input. It replaces repeated keys, braces, and quotes with indentation and tabular rows. Its strongest use case is a uniform array of objects; deeply nested or irregular data can favor JSON, so measure representative payloads before adopting it.

Example payload in JSON:

Equivalent payload in TOON: 

In this TOON encoding, the header line users[2]{id,name,role}: declares the array once: [2] is the number of rows that follow, and {id,name,role} names the fields shared by every row. Each user then collapses to a single comma-separated line instead of repeating the keys, braces, and quotes. TOON's documentation reports about 117 tokens for this style of sample data in JSON versus about 66 in TOON, but savings depend on data shape, formatting, and tokenizer.

Compact as that looks, keep JSON as the default and treat TOON as a narrow exception. JSON is universally supported, dominates model training data, and stays easy to inspect and filter with standard tooling such as jq. Reserve TOON for large, uniform arrays where representative benchmarks show real token savings without an accuracy penalty—and use it only at the LLM boundary, not as your canonical application or API format.

For coding agents, this is the common case rather than the exception: they run in long, multi-turn sessions. A benchmark study of token-optimized formats in agentic systems found that TOON cut tokens by up to 18% but at a roughly 9-percentage-point accuracy cost, and that it "loses accuracy further in multi-turn settings, where parsing failures cascade into additional reasoning iterations and erode per-call gains" (Kutschka & Geiger, 2026), concluding that TOON "is not safe as a default." The same study found TRON (Token Reduced Object Notation) safer—up to 27% fewer tokens within 14 points of JSON accuracy—when the workload exposes many structurally similar tool schemas. Measure end-to-end cost and task accuracy on your own workload before adopting either.

10. Use dedicated tools to optimize token usage carefully

Even well-scoped coding tasks can generate substantial context: test results, compiler output, repository scans, logs, structured payloads, tool responses, and earlier messages. A growing ecosystem of tools promises to reduce this overhead by compressing, filtering, or reshaping the information that reaches the model. Examples include RTK, Headroom, Context Mode, Caveman, Ponytail, repository graph tools, symbol indexes, and compact data formats.

These tools can be useful, but they should not be treated as a universal cost-saving switch. Advertised savings of 33–99% per compressed payload can be technically real, while the impact on the full bill remains small. In complete coding-agent sessions, many tokens sit outside the compressible path: repository context, tool definitions, prompt-cache reads and writes, conversation history, reasoning overhead, generated code, and file diffs.

This is why compression effects are often diluted over time. In a short, output-heavy session, a tool that removes verbose logs or conversational filler may save a visible share of tokens. In a long coding session, however, the savings can be washed out by repeated retrieval, cache activity, accumulated history, and non-compressible code or tool traffic. A useful independent data point comes from an investigation of roughly 500 Claude Code sessions with 614 million processed tokens and a baseline spend of about $926. Although some tools advertised large per-payload reductions, the measured real-bill impact was modest: Headroom saved 2.8%, RTK 0.5%, Caveman 0.4%, and all tools combined 3.7%. The conclusion is not that these tools are useless, but that headline compression ratios rarely translate directly into equivalent end-to-end cost reduction.

Recent JetBrains benchmarks tell the same story in a controlled setting. Caveman advertised a 65% output-token reduction, but measured savings on realistic Claude Code agent tasks were 8.5%. A separate JetBrains RTK benchmark found that RTK’s advertised 60–90% token reduction did not translate into lower end-to-end agent cost: on real Claude Code agent work, RTK was 7.6% more expensive at low reasoning effort and showed no measurable cost difference at high reasoning effort, while task quality remained unchanged. Ponytail was more promising because it tries to reduce unnecessary code generation itself: JetBrains measured about 15% less code, 10.3% lower cost, and 11% less time, still well below the advertised figures. These results are task-specific and depend on the benchmark harness, model, repository, and assistant architecture.

Security and governance deserve special attention. Tools such as RTK intercept or transform command output before it is shown to the model. That can remove noise, but it can also remove evidence. Over-aggressive filtering may hide warnings, stack traces, failing checks, policy violations, secret-detection findings, or other security-relevant signals. In regulated or security-sensitive environments, teams should preserve raw logs for auditability, test filters on representative failures, and avoid compressing outputs from security scans or compliance checks unless the original details remain available.

These investigations are also specific to their harness and assistant architecture. Claude Code, GitHub Copilot, Cursor, Aider, OpenCode, and other coding assistants differ in how they manage context, tools, caching, repository access, and agent loops. A tool that helps in one environment may have less impact, no impact, or different risks in another. Therefore, teams should benchmark token tools with their own repositories, task mix, security requirements, and quality gates before adopting them broadly.

The practical recommendation is cautious: use these tools selectively for short, output-heavy sessions or clearly identified bottlenecks, not as a substitute for good task scoping, selective retrieval, deterministic workflows, and session hygiene. The most reliable strategy is still to prevent irrelevant context from entering the session in the first place.

Conclusion

Working token-efficiently with AI coding assistants is primarily a context-engineering problem - and the same discipline that lowers cost also shortens delivery time, because an agent that receives the smallest sufficient context reaches a correct, verifiable result in fewer round trips. The largest gains come from the top of the list: scoping each task precisely, matching the model and its reasoning effort to the work, and separating planning from execution. Workflow and session hygiene - deterministic checks, timely compaction, and a stable prompt cache - compound those gains, while curated instruction files, English defaults, language and format choices, and dedicated compression tools are secondary levers whose headline savings should be validated against real repositories, languages, and quality requirements.

The full set of habits, ordered by leverage:

• Write precise, scoped prompts that name files, symbols, failing tests, and acceptance criteria - not a long narrative.
• Match the model and its reasoning effort to the task; test a lighter model at high or max reasoning before upgrading to a larger one.
• Separate planning from execution: plan with a reasoning model, then run the approved plan in a fresh session with a cheaper model.
• Let deterministic tools (tests, linters, scans, CI) judge correctness rather than asking the model to verify its own work.
• Compact at milestones and restart when the goal changes, carrying forward a structured handover of objective, decisions, files, test status, and risks.
• Keep the session configuration stable - model, reasoning effort, and tools - so the prompt cache stays warm during a task.
• Keep AGENTS.md short (~200 lines), human-owned, and free of auto-generated filler.
• Keep repeatedly-loaded context in English - AGENTS.md, skills, docstrings, and prompt templates.
• Treat programming-language and data-format token efficiency as a secondary factor - for example, TOON only at the LLM boundary - validated against real needs.
• Use dedicated compression and retrieval tools selectively, and benchmark them on your own repositories before adopting them broadly.

The most useful principle is simple:

Provide the smallest context that is sufficient to make the next correct, verifiable decision.

Teams that apply this principle reduce both cost and latency while making agent behavior easier to understand, review, and govern.

References and Further Reading

1. Sahajmeet Kaur, TrueFoundry,“OpenCode Token Usage: How It Works and How to Optimize It,” 27 July 2026. https://www.truefoundry.com/blog/opencode-token-usage-how-it-works-and-how-to-optimize-it

2. Pochi,“Five Practical Tips to Save Token Consumption with Pochi,” publication date not stated, accessed 28 July 2026. https://docs.getpochi.com/developer-updates/reduce-token-consumption-with-pochi/

3. Aleksandar Petrov, Emanuele La Malfa, Philip H. S. Torr and Adel Bibi,“Language Model Tokenizers Introduce Unfairness Between Languages,” NeurIPS 2023. https://arxiv.org/pdf/2305.15425

4. Simiao Ren et al.,“Mythbuster: Chinese Language Is Not More Efficient Than English in Vibe Coding: A Preliminary Study on Token Cost and Problem-Solving Rate,” arXiv:2604.14210v1, 6 April 2026. https://arxiv.org/html/2604.14210v1

5. GitHub Docs,“Optimizing Your AI Usage to Maximize Efficiency and Reduce Cost,” publication date not stated, accessed 28 July 2026. Features and availability may vary by Copilot client, plan, model, and product version. https://docs.github.com/en/copilot/tutorials/optimize-ai-usage

6. RTK contributors,“RTK: High-Performance CLI Proxy,” GitHub project documentation, publication date not stated, accessed 28 July 2026. https://github.com/rtk-ai/rtk

7. Yamada Shun and contributors,“Repomix,” GitHub project documentation, publication date not stated, accessed 28 July 2026. https://github.com/yamadashy/repomix

8. Colby McHenry and contributors,“CodeGraph,” GitHub project documentation, publication date not stated, accessed 28 July 2026. https://github.com/colbymchenry/codegraph

9. DeusData and contributors,“codebase-memory-mcp,” GitHub project documentation, publication date not stated, accessed 28 July 2026. https://github.com/DeusData/codebase-memory-mcp

10. J. Gravelle and contributors,“jCodeMunch MCP,” GitHub project documentation, publication date not stated, accessed 28 July 2026. https://github.com/jgravelle/jcodemunch-mcp

11. Graphify,“Knowledge Graphs for AI Coding Assistants,” publication date not stated, accessed 28 July 2026. https://graphify.net/knowledge-graph-for-ai-coding-assistants.html

12. Headroom Labs and contributors,“Headroom,” GitHub project documentation, publication date not stated, accessed 28 July 2026. https://github.com/headroomlabs-ai/headroom

13. Johann Schopplich and contributors, “TOON: Getting Started,” TOON documentation, version 4.1.0, publication date not stated, accessed 29 July 2026. https://toonformat.dev/guide/getting-started.html

14. Context-mode contributors,“Context-Mode: Context Window Optimization for AI Coding Agents,” GitHub project documentation, publication date not stated, accessed 28 July 2026. https://github.com/mksglu/context-mode

15. Anthropic,“Manage Costs Effectively,” Claude Code documentation, publication date not stated, accessed 28 July 2026. Product behavior may vary by version. https://code.claude.com/docs/en/costs#reduce-token-usage

16. Julius Brussee and contributors,“Caveman,” GitHub project documentation, publication date not stated, accessed 28 July 2026. The stated token reduction is a project claim and is not treated as independently verified in this article. https://github.com/juliusbrussee/caveman

17. Martin Alderson,“Which Programming Languages Are Most Token-Efficient?” 8 January 2026. The author describes the analysis as exploratory rather than a scientific study. https://martinalderson.com/posts/which-programming-languages-are-most-token-efficient/

18. The Net Revenue,“How Much Money Is It Costing You Not to Write to AI in English?” publication date not stated, accessed 28 July 2026. https://www.thenetrevenue.com/en/tokens-language-english-ai/

19. Denis Shiryaev, JetBrains,“Speaking to AI Agents like Cavemen Saves 65% of Tokens. We Test,” 6 July 2026. https://blog.jetbrains.com/ai/2026/07/speak-to-ai-agents-like-cavemen-tosave-tokens/

20. Denis Shiryaev, JetBrains,“Ponytail Skill for Claude Code: Does It Really Cut Tokens,” 28 July 2026. https://blog.jetbrains.com/ai/2026/07/ponytail-skill-claude-tested/

21. Denis Shiryaev, JetBrains,“rtk Claude Code Token Savings: A Skill Trial Benchmark,” 20 July 2026. https://blog.jetbrains.com/ai/2026/07/rtk-claude-code-token-savings/

22. CodePointer,“Cutting LLM Token Costs with RTK,” publication date not stated, accessed 29 July 2026. https://codepointer.substack.com/p/cutting-llm-token-costs-with-rtk

23. Thibaud Gloaguen, Niels Mündler, Mark Müller, Veselin Raychev and Martin Vechev, “Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?” arXiv:2602.11988v1, 12 February 2026. https://arxiv.org/html/2602.11988v1

24. Token Calculator, “Most Token-Efficient Languages for LLMs, Ranked & Priced,” publication date not stated, accessed 30 July 2026. https://tokencalculator.ai/most-token-efficient-languages-for-llms-ranked-priced/

25. Niccolò Marini, Leonardo Pampaloni, Filippo Di Martino, Roberto Verdecchia and Enrico Vicario, “Green AI: Which Programming Language Consumes the Most?” 9th International Workshop on Green and Sustainable Software (GREENS), 2025. https://robertoverdecchia.github.io/papers/GREENS\_2025.pdf

26. DietrichGebert and contributors, "Ponytail," GitHub project documentation, publication date not stated, accessed 30 July 2026. https://github.com/DietrichGebert/ponytail

27. Artificial Analysis, "AI Coding Agent Benchmarks & Leaderboard," publication date not stated, accessed 30 July 2026. https://artificialanalysis.ai/agents/coding-agents

28. Anthropic, "How Claude Code Uses Prompt Caching," Claude Code documentation, publication date not stated, accessed 30 July 2026. Product behavior may vary by version, model, and provider. https://code.claude.com/docs/en/prompt-caching

29. Janakiram MSV, Forbes, "Uber Burns Its 2026 AI Budget in Four Months on Claude Code," 17 May 2026. https://www.forbes.com/sites/janakirammsv/2026/05/17/uber-burns-its-2026-ai-budget-in-four-months-on-claude-code

30. Efosa Udinmwen, TechRadar Pro, "Amazon Admits It Accidentally Shelled Out $1.8 Million for Claude to Finish Its Menial Coding Tasks," accessed 7 August 2026. https://www.techradar.com/pro/amazon-admits-it-accidentally-shelled-out-usd1-8-million-for-claude-to-finish-menial-coding-tasks

31. Lorenz Kutschka and Bernhard C. Geiger, "Notation Matters: A Benchmark Study of Token-Optimized Formats in Agentic AI Systems," arXiv:2605.29676v2, 17 June 2026. https://arxiv.org/html/2605.29676v2

32. Timothy Huang and contributors, "TRON: Token Reduced Object Notation (JavaScript library)," GitHub project documentation, publication date not stated, accessed 7 August 2026. https://github.com/tron-format/tron-javascript