Structured Generations (JSON)

Cohere models such as Command R and Command R+ are great at producing structured outputs in formats such as JSON.

Why generate JSON Objects using an LLM?

JSON is a lightweight format that is easy for humans to read and write and is also easy for machines to parse. By generating JSON objects, you can structure and organize the model’s responses in a way that can be used in downstream applications. This is particularly useful when you want to extract specific information from the responses, perform data analysis, or integrate the responses into your applications seamlessly.

How to use the response_format parameter

When making an API request, you can specify the response_format parameter to indicate that you want the response in a JSON object format.

1
import cohere
2
3
co = cohere.ClientV2(api_key="YOUR API KEY")
4
5
res = co.chat(
6
    model="command-r-plus-08-2024",
7
    messages=[
8
        {
9
            "role": "user",
10
            "content": "Generate a JSON describing a person, with the fields 'name' and 'age'",
11
        }
12
    ],
13
    response_format={"type": "json_object"}
14
)
15
16
print(res.message.content[0].text)

By setting the response_format type to "json_object" in the Chat API, the output of the model is guaranteed to be a valid JSON object.

# Example response
{
  "name": "Emma Johnson",
  "age": 32
}

Specifying a schema (beta)

The response_format parameter also allows you to define a schema for the generated JSON object. A JSON Schema is a way to describe the structure of the JSON object you want the LLM to generate. This is optional, but it gives you more control over the response format.

For example, let’s say you want the LLM to generate a JSON object with specific keys for a book, such as “title,” “author,” and “publication_year.” Your API request might look like this:

1
import cohere
2
3
co = cohere.ClientV2(api_key="YOUR API KEY")
4
5
res = co.chat(
6
    model="command-r-plus-08-2024",
7
    messages=[
8
        {
9
            "role": "user",
10
            "content": "Generate a JSON describing a book, with the fields 'title' and 'author' and 'publication_year'",
11
        }
12
    ],
13
    response_format={
14
        "type": "json_object",
15
        "schema": {
16
            "type": "object",
17
            "required": ["title", "author", "publication_year"],
18
            "properties": {
19
                "title": {"type": "string"},
20
                "author": {"type": "string"},
21
                "publication_year": {"type": "integer"},
22
            },
23
        },
24
    }
25
)
26
27
print(res.message.content[0].text)

In this schema, we defined three keys (“title,” “author,” “publication_year”) and their expected data types (“string” and “number”). The LLM will generate a JSON object that adheres to this structure.

# Example response
{
  "title": "The Great Gatsby",
  "author": "F. Scott Fitzgerald",
  "publication_year": 1925
}

Generating nested objects

By setting response_format={ "type": "json_object" }the model can be configured to output objects with up to 5 levels of nesting. When a schema is specified, there are no limitations on the levels of nesting.

Schema constraints

When constructing a schema keep the following constraints in mind:

• The type in the top level schema must be object
• Every object in the schema must have at least one required field specified

Unsupported schema features

We do not support the entirety of the JSON Schema specification. Below is a list of some unsupported features:

• Schema Composition (anyOf, allOf, oneOf and not)
• Numeric Ranges (maximum and minimum)
• Array Length Ranges (minItems and maxItems)
• String limitations:
  • String Length (maxLength and minLength)
  • The following are not supported in Regular Expressions
    • ^
    • $
    • ?=
    • ?!
  • The following formats are the only supported ones
    • date-time
    • uuid
    • date
    • time