Unveiling the Hidden Costs of AI: Are Token-Based Billing Practices Overcharging Users?

Recent studies reveal that the token-based billing model used by AI service providers obscures the true costs for consumers. By manipulating token counts and embedding hidden processes, companies can subtly inflate billing amounts. Although auditing tools are suggested, inadequate oversight leaves users unaware of the excessive charges they incur.

Understanding AI Billing: The Role of Tokens

Today, most consumers using AI-driven chat services, like ChatGPT-4o, are billed based on tokens—invisible text units that go unnoticed yet affect cost dramatically. While exchanges are priced according to token consumption, users lack direct access to verify token counts.

Despite a general lack of clarity about what we are getting for our token purchases, this billing method has become ubiquitous, relying on a potentially shaky foundation of trust.

What are Tokens and Why Do They Matter?

A token isn’t quite equivalent to a word; it includes words, punctuation, or fragments. For example, the word ‘unbelievable’ might be a single token in one system but split into three tokens in another, inflating charges.

This applies to both user input and model responses, with costs determined by the total token count. The challenge is that users are not privy to this process—most interfaces do not display token counts during conversations, making it nearly impossible to ascertain whether the charges are fair.

Recent studies have exposed serious concerns: one research paper shows that providers can significantly overcharge without breaking any rules, simply by inflating invisible token counts; another highlights discrepancies between displayed and actual token billing, while a third study identifies internal processes that add charges without benefiting the user. The result? Users may end up paying for more than they realize, often more than expected.

Exploring the Incentives Behind Token Inflation

The first study, titled Is Your LLM Overcharging You? Tokenization, Transparency, and Incentives, argues that the risks associated with token-based billing extend beyond simple opacity. Researchers from the Max Planck Institute for Software Systems point out a troubling incentive for companies to inflate token counts:

‘The core of the problem lies in the fact that the tokenization of a string is not unique. For instance, if a user prompts “Where does the next NeurIPS take place?” and receives output “|San| Diego|”, one system counts it as two tokens while another may inflate it to nine without altering the visible output.’

The paper introduces a heuristic that can manipulate tokenization without altering the perceived output, enabling measurable overcharges without detection. The researchers advocate for a shift to character-based billing to foster transparency and fairness.

Addressing the Challenges of Transparency

The second paper, Invisible Tokens, Visible Bills: The Urgent Need to Audit Hidden Operations in Opaque LLM Services, expands on the issue, asserting that hidden operations—including internal model calls and tool usage—are rarely visible, leading to misaligned incentives.

These factors contribute to structural opacity, where users are charged based on unverifiable metrics. The authors identify two forms of manipulation: quantity inflation, where token counts are inflated without user benefit, and quality downgrade, where lower-quality models are used without user knowledge.

Counting the Invisible: A New Perspective

The third paper from the University of Maryland, CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIs, reframes the issue of billing as structural rather than due to misuse or misreporting. It highlights that most commercial AI services conceal intermediate reasoning while charging for it.

‘This invisibility allows providers to misreport token counts or inject fabrications to inflate charges.’

CoIn employs cryptographic verification methods and semantic checks to detect token inflation, achieving a detection success rate nearing 95%. However, this framework still relies on voluntary cooperation from providers.

Conclusion: A Call for Change in AI Billing Practices

Token-based billing can obscure the true value of services, much like a scrip-based currency shifts consumer focus away from actual costs. With the intricate workings of tokens hidden, users risk being misled about their spending.

Although character-based billing could offer a more transparent alternative, it could also introduce new discrepancies based on language efficiency. Overall, without legislative action, it appears unlikely that consumers will see meaningful reform in how AI services bill their usage.

First published Thursday, May 29, 2025