How to Evaluate your LLM Response (v1 API)

You can leverage Command R to evaluate natural language responses that cannot be easily scored with manual rules.

Prompt

You are an AI grader that given an output and a criterion, grades the completion based on the prompt and criterion. Below is a prompt, a completion, and a criterion with which to
grade the completion. You need to respond according to the criterion instructions.
## Output
The customer's UltraBook X15 displayed a black screen, likely due to a graphics driver issue.
Chat support advised rolling back a recently installed driver, which fixed the issue after a
system restart.
## Criterion 
Rate the ouput text with a score between 0 and 1. 1 being the text was written in a formal
and business appropriate tone and 0 being an informal tone. Respond only with the score.

Output

0.8

API Request

PYTHON

1 import cohere
2 
3 co = cohere.Client(api_key="Your API key")
4 response = co.chat(
5     message="""
6 You are an AI grader that given an output and a criterion, grades the completion based on
7 the prompt and criterion. Below is a prompt, a completion, and a criterion with which to grade
8 the completion. You need to respond according to the criterion instructions.
9 
10 ## Output
11 The customer's UltraBook X15 displayed a black screen, likely due to a graphics driver issue.
12 Chat support advised rolling back a recently installed driver, which fixed the issue after a
13 system restart.
14 
15 ## Criterion 
16 Rate the ouput text with a score between 0 and 1. 1 being the text was written in a formal
17 and business appropriate tone and 0 being an informal tone. Respond only with the score.
18 """,
19 )
20 print(response)

1	import cohere
2
3	co = cohere.Client(api_key="Your API key")
4	response = co.chat(
5	message="""
6	You are an AI grader that given an output and a criterion, grades the completion based on
7	the prompt and criterion. Below is a prompt, a completion, and a criterion with which to grade
8	the completion. You need to respond according to the criterion instructions.
9
10	## Output
11	The customer's UltraBook X15 displayed a black screen, likely due to a graphics driver issue.
12	Chat support advised rolling back a recently installed driver, which fixed the issue after a
13	system restart.
14
15	## Criterion
16	Rate the ouput text with a score between 0 and 1. 1 being the text was written in a formal
17	and business appropriate tone and 0 being an informal tone. Respond only with the score.
18	""",
19	)
20	print(response)