Structured Outputs

Overview

Structured Outputs is a feature for forcing the LLM’s output to follow a specified format 100% of the time. This increases the reliability of LLM in enterprise applications where downstream applications expect the LLM output to be correctly formatted. By forcing the model to follow a structured schema, hallucinated fields and entries in structured data can be reliably eliminated.

Compatible models:

• Command R 08 2024
• Command R
• Command R+ 08 2024
• Command R+

How to Use Structured Outputs

There are two ways to use Structured Outputs

• Structured Outputs (JSON). This is primarily used in text generation use cases.
• Structured Outputs (Tools). This is primarily used in tool use and agents use cases via function calling.

Structured Outputs (JSON)

Here, you can call the Chat API to generate Structured Outputs in JSON format. JSON is a lightweight format that is easy for humans to read and write and is also easy for machines to parse.

This is particularly useful in text generation use cases, for example, when you want to extract specific information from the responses, perform data analysis, or integrate the responses into your applications seamlessly.

There are two ways of specifying the JSON output:

• JSON mode
• JSON Schema mode

JSON mode

In JSON mode, when making an API request, you can specify the response_format parameter to indicate that you want the response in a JSON object format.

1
import cohere
2
co = cohere.Client(api_key="YOUR API KEY")
3
4
res = co.chat(
5
  model="command-r-plus-08-2024",
6
  message="Generate a JSON describing a person, with the fields 'name' and 'age'",
7
  response_format={ "type": "json_object" }
8
)
9
10
print(res.text)

By setting the response_format type to "json_object" in the Chat API, the output of the model is guaranteed to be a valid JSON object.

# Example response
{
  "name": "Emma Johnson",
  "age": 32
}

JSON Schema mode

In JSON Schema mode, you can optionally define a schema as part of the response_format parameter. A JSON Schema is a way to describe the structure of the JSON object you want the LLM to generate.

This forces the LLM to stick to this schema, thus giving you greater control over the output.

For example, let’s say you want the LLM to generate a JSON object with specific keys for a book, such as “title,” “author,” and “publication_year.” Your API request might look like this:

1
import cohere
2
co = cohere.Client(api_key="YOUR API KEY")
3
4
res = co.chat(
5
  model="command-r-plus-08-2024",
6
  message="Generate a JSON describing a book, with the fields 'title' and 'author' and 'publication_year'",
7
  response_format={
8
    "type": "json_object",
9
    "schema": {
10
      "type": "object",
11
      "required": ["title", "author", "publication_year"],
12
      "properties": {
13
        "title": { "type": "string" },
14
        "author": { "type": "string" },
15
        "publication_year": { "type": "integer" }
16
      }
17
    }
18
  }
19
)
20
21
print(res.text)

In this schema, we defined three keys (“title,” “author,” “publication_year”) and their expected data types (“string” and “number”). The LLM will generate a JSON object that adheres to this structure.

# Example response
{
  "title": "The Great Gatsby",
  "author": "F. Scott Fitzgerald",
  "publication_year": 1925
}

Specifying a schema

Generating nested objects

The model can be configured to output objects with up to 5 levels of nesting. When a schema is specified, there are no limitations on the levels of nesting.

Schema constraints

When constructing a schema keep the following constraints in mind:

• The type in the top level schema must be object
• Every object in the schema must have at least one required field specified

Unsupported schema features

We do not support the entirety of the JSON Schema specification. Below is a list of some unsupported features:

• Schema Composition (anyOf, allOf, oneOf and not)
• Numeric Ranges (maximum and minimum)
• Array Length Ranges (minItems and maxItems)
• String limitations:
  • String Length (maxLength and minLength)
  • The following are not supported in Regular Expressions
    • ^
    • $
    • ?=
    • ?!
  • The following formats are the only supported ones
    • date-time
    • uuid
    • date
    • time
• Others:
  • uniqueItems
  • additionalProperties