JSON vs TOON: Experimenting with in LLM-Optimized Data Formats

[Anita_Okoh]

Introduction

At some point in every LLM project, we do the same thing: we take a dataset, serialize it as JSON, paste it into a prompt, and quietly hope the model “figures it out”.

For small samples, it usually does.

But as the data grows, things get fuzzy.

JSON has become the default data input format for LLM prompts, not because it is optimal, but because it is familiar. It is deeply embedded in APIs, storage systems, and validation pipelines. It is readable, ubiquitous, and already part of almost every data workflow.  JSON is structured, explicit, and easy to understand (readable), which explains why it became the default in the first place.

However, LLMs do not parse JSON the way traditional programs do. They tokenize it. That distinction matters because every repeated key, quote, and brace consumes context and costs without necessarily adding semantic value for the model.

While JSON remains the universal standard for APIs, a new contender, "TOON," has emerged to specifically address the token-heavy nature of JSON in AI workflows.

TOON is an emerging serialization format designed for LLM prompts. Its primary goal is token efficiency: minimizing the number of tokens required to represent structured data without losing schema information. TOON enters this space with a narrow but important claim: structured data can be represented in a way that is cheaper and easier for LLMs to process, even if it is less ideal for traditional software tooling.

This article does not argue that TOON should replace JSON.

Instead, it asks a simpler and more practical question:

What actually changes when we feed the same task and the same data to an LLM using JSON versus TOON?

To answer this, I ran a small experiment to examine three things: whether LLMs can correctly interpret TOON, how much token usage changes, and how the input format affects latency.

The Experiment

To measure both data formats fairly, the dataset, the initial prompt, and the user tasks were kept exactly the same. The only variable was the data format itself. 

• For the dataset, I used Python’s Faker library to generate 300 rows of product data. The same data was then serialized into both JSON and TOON. This is what the first row looks like
  • JSON

{
    "id": 1,
    "uuid": "b4a1a3a9-944b-48e9-9f59-3e0409b56e3e",
    "sku": "EDN-40866175",
    "name": "TechCorp Classic Board Game",
    "category": "Toys",
    "description": "Air service station tree gas individual factor. Science know it whether return some happen executive. Cut my treatment sometimes.",
    "price": 1690.82,
    "currency": "USD",
    "stock": 422,
    "manufacturer": "SmartBrand",
    "barcode": "0649893317279",
    "weight_kg": 32.96,
    "rating": 2.7,
    "reviews": 4818,
    "is_active": true,
    "created_at": "2025-12-29T14:12:29.674107"
  }​

• TOON

id: 1
uuid: b4a1a3a9-944b-48e9-9f59-3e0409b56e3e
sku: EDN-40866175
name: TechCorp Classic Board Game
category: Toys
description: Air service station tree gas individual factor. Science know it whether return some happen executive. Cut my treatment sometimes.
price: 1690.82
currency: USD
stock: 422
manufacturer: SmartBrand
barcode: 0649893317279
weight_kg: 32.96
rating: 2.7
reviews: 4818
is_active: true
created_at: "2025-12-29T14:12:29.674107"​

• Three tasks were used to evaluate the formats, which are:
  • What is the price of the product with id 42? (a lookup task)
  • What are all the IDs of products that are phones? (a filtering task)
  • Return all the data for the product with id 121. (a record retrieval task)
• Other settings: gpt 4o model (default settings), latency was measured using the Python time library before and after the API call, and only the input tokens were counted using tiktoken (o200k_base encoding)

To reduce randomness, I ran 10 trials per format (30 queries each format: 10 × 3 tasks). Execution order was alternated to reduce the chance that the transient API load influenced only one format.

The Results

From the results above, a few clear observations emerged.

In terms of accuracy, both JSON and TOON returned correct results across all tasks. This indicates that the LLM was able to correctly interpret the TOON structure and extract the required information just as reliably as it did with JSON.

In terms of token count, TOON used approximately 22 percent fewer tokens than JSON for the same dataset. This reduction came from eliminating repeated structural elements present in JSON.

Latency showed a similar pattern. On average, prompts using JSON took around 36 seconds to complete, while the TOON version took approximately 22 seconds. This represents roughly a 36 percent improvement in runtime. When normalized, this corresponds to about 0.50 seconds per 1,000 tokens for TOON, compared to 0.61 seconds per 1,000 tokens for JSON.

Conclusion

These results are not conclusive proof that TOON is better than JSON in all cases. JSON structures can become deeply complex, and it is unclear how far TOON can scale before readability or model comprehension becomes an issue. It is also worth noting that LLMs have mostly been trained on significantly more JSON formats than TOONs.

What this experiment suggests is not a replacement of JSON with TOON, but a more nuanced takeaway: TOON can be a promising alternative at the prompt layer, particularly when mid-sized structured datasets need to be injected into LLM prompts and token optimization, or speed is a concern.

JSON remains the right choice for systems and tooling. TOON may be a useful option for models.