TOON vs. JSON

• TOON formatted data can be used instead of JSON formatted data in calls to LLMs. TOON claims to be able to reduce your input token usage by ~40%.

• TOON formatted data has also been shown to increase the accuracy of prompts. This only applies to structured data formats rather than prompts that have been converted from plain text to JSON.

• TOON cannot replace JSON in model outputs. For parsing, function calling, and other cases where JSON outputs are useful, TOON has not been shown to be effective yet.

There are two main reasons people use JSON formatting in their prompts to an LLM. First, some developers replace plain text prompting with JSON formatted prompting because they prefer its clear and consistent layout.

Example
json_prompt = {
  "task": "summarize_email",
  "email": "Love the app, but it keeps crashing. It shows me a black screen with no words appearing. I've tried logging out and back in again.",
  "output": "list",
  "max_points": 3
}

The above JSON can be converted to a Python string and passed through an LLM to get the following response. This is an alternative to writing out plain text instructions explaining the LLM’s role and providing it with instructions to summarize the email with three points in a list.

Output
 - The user is experiencing repeated app crashes, specifically encountering a black screen without text.

 - They have attempted troubleshooting by logging out and back in without success.

 - The user expresses overall satisfaction with the app despite the prevailing issue.

Replacing plain text with JSON formatted instructions may work better for specific architectures, but it is not clear that JSON is better than plain text in most cases. The number of tokens can often be greater than with plain text, and results from formal studies haven’t shown better accuracy. In one such study comparing plain text, markdown, JSON, and YAML, results indicated that the accuracy can vary drastically from model to model, and no single format was a clear winner. Whether or not to replace plain text prompts with JSON should be decided based on the needs of the specific application by testing with your model and your use-case.

The second reason to use JSON is if you need to include a large set of structured data in your prompt context for the LLM.

JSON_dataset = {
  "animals": [
    {
      "name": "African Elephant",
      "category": "mammal",
      "habitat": "savanna",
      "diet": "herbivore"
    },
    {
      "name": "Red-Eyed Tree Frog",
      "category": "amphibian",
      "habitat": "rainforest",
      "diet": "carnivore"
    },
    {
      "name": "Bald Eagle",
      "category": "bird",
      "habitat": "forests near water",
      "diet": "carnivore"
    },
    {
      "name": "Great White Shark",
      "category":  "fish",
      "habitat":  "ocean",
      "diet":  "carnivore"
    }
  ]
}

prompt = f"Based on their habitat and diet, which of the following animals would be most difficult to keep in a zoo? Respond with only the name of the animal. {JSON_dataset}"

In the prompt above, some example database entries are passed along with a text prompt to an LLM. This allows the LLM to do a logic-based query on a portion of the dataset. In this case, the LLM is being asked which of the animals would be most difficult to keep in a zoo based on the attributes included in the database. The output is displayed below.

Output
Great White Shark.

In this use-case, the JSON context was not converted from plain text and is coming from a source that already had structured data. The data is usually then included with plain text instructions. These use-cases are where TOON data formats may come in useful for you.

The main reason to use TOON is to get the benefits of JSON or other structured data while using fewer tokens. If encode our JSON example from above into a TOON data format, we can see the number of tokens has decreased significantly because it is less verbose and has fewer repeated tokens with a similar output.

TOON_dataset = """animals[4]{name,category,habitat,diet}:
  African Elephant,mammal,savanna,herbivore
  Red-Eyed Tree Frog,amphibian,rainforest,carnivore
  Bald Eagle,bird,forests near water,carnivore
  Great White Shark,fish,ocean,carnivore"""

prompt = f"Based on their habitat and diet, which of the following animals would be most difficult to keep in a zoo? Respond with only the name of the animal. {TOON_dataset}"

The dataset above contains the same information as the previous JSON, but it has been converted into the TOON format. As you can see, it uses fewer tokens (71) compared to the JSON’s 172, which is a 59% reduction. Furthermore, the output below has the same result as the JSON one.

Output
Great White Shark

On the GitHub release page, TOON is shown to have higher accuracy than JSON while using ~40% fewer tokens across four different model providers. They also offer a Python converter that you can use to convert the JSON into a TOON format before passing the data to an LLM. We will cover how to use it below.

To determine the best prompting format for your application, start by determining whether or not you will need to pass structured data from a database or some other source to the LLM as JSON, CSV, XML, YAML, etc. If you do not have structured data and your prompts are already in plain language, TOON is probably not your best choice. Continue to use plain language prompts and adjust them to increase performance.

If you do have structured data that you are trying to pass, then it could be worth it for your application to test out TOON formatted prompts. With all prompt formatting options, TOON included, the most quantifiable way to measure a format’s value to your application is by counting the number of tokens used. Test different options, narrow down which ones meet your accuracy needs, then choose the option that uses the fewest tokens. In this tutorial, you will use a TOON Python package to create a workflow that converts JSON into TOON before sending it to an LLM endpoint.

First, set up your environment by downloading Python and installing the TOON Python library.

pip install git+https://github.com/toon-format/toon-python.git

Create a Python script called TOON_demo.py. Ensure you have access to a large language model API. This tutorial uses a Mistral 3 model that was deployed on a DigitalOcean GPU Droplet following the Mistral 3 deployment tutorial.

Next, you will import the Python dependencies and write the LLM calling function. Replace the highlighted your_server_ip placeholder value with the Public IPv4 of your instance. Alternatively, you can change this function to use an LLM API service.

import requests
import json
from toon_format import encode, decode

url = "http://your_server_ip:8000/v1/completions"

def call_LLM(dataset):
    prompt = f"Based on their habitat and diet, which of the following animals would be most difficult to keep in a zoo? Respond with only the name of the animal. {dataset}"

    data = {
        "model": "mistralai/Ministral-3-14B-Instruct-2512",
        "prompt": prompt,
        "max_tokens": 500
    }

    response = requests.post(url, json=data)
    response_message = response.json()['choices'][0]['text']
    print(response_message)

Next, use the encode function from the toon_format library to convert your JSON before making your LLM call.

JSON_dataset = {
  "animals": [
    {
      "name": "African Elephant",
      "category": "mammal",
      "habitat": "savanna",
      "diet": "herbivore"
    },
    {
      "name": "Red-Eyed Tree Frog",
      "category": "amphibian",
      "habitat": "rainforest",
      "diet": "carnivore"
    },
    {
      "name": "Bald Eagle",
      "category": "bird",
      "habitat": "forests near water",
      "diet": "carnivore"
    },
    {
      "name": "Great White Shark",
      "category":  "fish",
      "habitat":  "ocean",
      "diet":  "carnivore"
    }
  ]
}

TOON_dataset = encode(JSON_dataset)
print(TOON_dataset)

Encoding the JSON dataset returns a TOON formatted string.

Output
animals[4]{name,category,habitat,diet}:
  African Elephant,mammal,savanna,herbivore
  Red-Eyed Tree Frog,amphibian,rainforest,carnivore
  Bald Eagle,bird,forests near water,carnivore
  Great White Shark,fish,ocean,carnivore

Decoding the TOON dataset returns a “minified” JSON object without the newlines and whitespace that are normally in JSON.

JSON_formatted_dataset = decode(TOON_dataset)
print(TOON_dataset)

Output
{'animals': [{'name': 'African Elephant', 'category': 'mammal', 'habitat': 'savanna', 'diet': 'herbivore'}, {'name': 'Red-Eyed Tree Frog', 'category': 'amphibian', 'habitat': 'rainforest', 'diet': 'carnivore'}, {'name': 'Bald Eagle', 'category': 'bird', 'habitat': 'forests near water', 'diet': 'carnivore'}, {'name': 'Great White Shark', 'category': 'fish', 'habitat': 'ocean', 'diet': 'carnivore'}]}

Next, run and check for accurate results between your JSON and TOON datasets.

print("JSON Dataset")
call_LLM(JSON_dataset)

print("\n\nTOON Dataset")
call_LLM(TOON_dataset)

Output
JSON Dataset
Great White Shark

TOON Dataset
Great White Shark

The toon_format library has an estimate_savings function for checking the differences in tokens. Import the function and pass the JSON_dataset.

from toon_format import estimate_savings

savings = estimate_savings(JSON_dataset)
print(f"Estimated savings: {savings}")

Output
{'json_tokens': 169, 'toon_tokens': 71, 'savings': 98, 'savings_percent': 57.98816568047337}

Token counts can vary by model, but the function returns the estimated tokens for each data format and the percentage of savings.

Does TOON work with nested or complex JSON structures?

Yes, TOON can work with nested JSON structures. The TOON encoder from the toon-python library automatically converts nested objects and arrays into the TOON format. However, as your data structure becomes more complex with deeper nesting levels, you should test whether TOON maintains its token efficiency advantage and whether the model can still accurately interpret the data.

Can TOON be used for outputs?

You could try to get an LLM to return a TOON formatted output, in the same way that JSON outputs are used, then decode the TOON into JSON for parsing, but TOON has not existed long enough for most models to be trained on TOON inputs and outputs. JSON outputs can be useful for parsing, function calling, data access, and more, but aren’t as relevant in the context of TOON because models may have a harder time consistently outputting correctly formatted TOON strings. This might require some experimentation or fine tuning to get consistent results.

Are there any compatibility issues with different LLM models when using TOON?

TOON has been shown to work with a wide range of models and should work on any LLM that has been trained on diverse text data. Because it is relatively new, some models may not have seen as many examples of it during their training compared to JSON. Testing with your specific model is recommended.

Can I structure my plain text prompts in TOON and make requests in that format?

You could try to structure your plain text prompts in TOON and make instructive requests in that format, but as mentioned above, a study showed that converting plain text prompts to other structured formats, like JSON, did not result in more accurate results. Therefore, it is better to choose JSON or TOON when data that you need to include as context is already structured as JSON.

Do JSON or TOON prompts make outputs more deterministic?

This is debatable. Some believe JSON formatted prompts can make your LLM outputs more consistent or accurate. LLMs are inherently non-deterministic, meaning their outputs are going to be different each time. There are many attempts in the LLM community to try to get more consistency out of LLM applications, and JSON prompts are often sold as a way to do that. In reality, results vary drastically from user to user and from model to model, so to determine whether or not TOON formatted prompts can make your outputs more deterministic, you would need to experiment with your data and model and see if it achieves your desired results.

TOON formatted data can be a lower cost and more efficient alternative to JSON when you intend to include JSON data in the context of your LLM prompt. It uses ~40% fewer tokens with comparable, if not improved accuracy results. Considering the way LLMs process large quantities of data, it is likely that most formats that allow for fewer context tokens, while maintaining the information, will automatically increase accuracy. TOON is one example of this, but more examples will continue to come out in the future.

In this tutorial, you used a TOON encoder to convert JSON data and estimate how many tokens would be saved with the different format. The next steps are to implement it in your application and test to ensure the results are acceptable for your use-case and that the token usage is reduced.

• Prompt Engineering Best Practices: Tips, Tricks, and Tools

• Mistral 3 Models on DigitalOcean

• Few-Shot Prompting: Techniques, Examples, and Best Practices