Building an Agent with Cohere

Open in Colab

Tool use extends the ideas from RAG, where external systems are used to guide the response of an LLM, but by leveraging a much bigger set of tools than what’s possible with RAG. The concept of tool use leverages LLMs’ useful feature of being able to act as a reasoning and decision-making engine.

While RAG enables applications that can answer questions, tool use enables those that can automate tasks.

Tool use also enables developers to build agentic applications that can take actions, that is, doing both read and write operations on an external system.

In this tutorial, you’ll learn about:

  • Creating tools
  • Tool planning and calling
  • Tool execution
  • Response and citation generation
  • Multi-step tool use

You’ll learn these by building an onboarding assistant for new hires.

Setup

To get started, first we need to install the cohere library and create a Cohere client.

PYTHON
1# pip install cohere
2
3import cohere
4import json
5
6co = cohere.ClientV2(api_key="COHERE_API_KEY") # Get your free API key: https://dashboard.cohere.com/api-keys

Creating tools

The pre-requisite, before we can run a tool use workflow, is to set up the tools. Let’s create three tools:

  • search_faqs: A tool for searching the FAQs. For simplicity, we’ll not implement any retrieval logic, but we’ll simply pass a list of pre-defined documents, which are the FAQ documents we had used in the Text Embeddings section.
  • search_emails: A tool for searching the emails. Same as above, we’ll simply pass a list of pre-defined emails from the Reranking section.
  • create_calendar_event: A tool for creating new calendar events. Again, for simplicity, we’ll not implement actual event bookings, but will return a mock success event. In practice, we can connect to a calendar service API and implement all the necessary logic here.

Here, we are defining a Python function for each tool, but more broadly, the tool can be any function or service that can receive and send objects.

PYTHON
1# Create the tools
2def search_faqs(query):
3 faqs = [
4 {"text": "Reimbursing Travel Expenses: Easily manage your travel expenses by submitting them through our finance tool. Approvals are prompt and straightforward."},
5 {"text": "Working from Abroad: Working remotely from another country is possible. Simply coordinate with your manager and ensure your availability during core hours."}
6 ]
7 return faqs
8
9def search_emails(query):
10 emails = [
11 {"from": "it@co1t.com", "to": "david@co1t.com", "date": "2024-06-24", "subject": "Setting Up Your IT Needs", "text": "Greetings! To ensure a seamless start, please refer to the attached comprehensive guide, which will assist you in setting up all your work accounts."},
12 {"from": "john@co1t.com", "to": "david@co1t.com", "date": "2024-06-24", "subject": "First Week Check-In", "text": "Hello! I hope you're settling in well. Let's connect briefly tomorrow to discuss how your first week has been going. Also, make sure to join us for a welcoming lunch this Thursday at noon—it's a great opportunity to get to know your colleagues!"}
13 ]
14 return emails
15
16def create_calendar_event(date: str, time: str, duration: int):
17 # You can implement any logic here
18 return {"is_success": True,
19 "message": f"Created a {duration} hour long event at {time} on {date}"}
20
21functions_map = {
22 "search_faqs": search_faqs,
23 "search_emails": search_emails,
24 "create_calendar_event": create_calendar_event
25}

The second and final setup step is to define the tool schemas in a format that can be passed to the Chat endpoint. The schema must contain the following fields: name, description, and parameters in the format shown below.

This schema informs the LLM about what the tool does, and the LLM decides whether to use a particular tool based on it. Therefore, the more descriptive and specific the schema, the more likely the LLM will make the right tool call decisions.

Further reading:

PYTHON
1# Define the tools
2tools = [
3 {
4 "type": "function",
5 "function": {
6 "name": "search_faqs",
7 "description": "Given a user query, searches a company's frequently asked questions (FAQs) list and returns the most relevant matches to the query.",
8 "parameters": {
9 "type": "object",
10 "properties": {
11 "query": {
12 "type": "string",
13 "description": "The query from the user"
14 }
15 },
16 "required": ["query"]
17 }
18 }
19 },
20 {
21 "type": "function",
22 "function": {
23 "name": "search_emails",
24 "description": "Given a user query, searches a person's emails and returns the most relevant matches to the query.",
25 "parameters": {
26 "type": "object",
27 "properties": {
28 "query": {
29 "type": "string",
30 "description": "The query from the user"
31 }
32 },
33 "required": ["query"]
34 }
35 }
36 },
37 {
38 "type": "function",
39 "function": {
40 "name": "create_calendar_event",
41 "description": "Creates a new calendar event of the specified duration at the specified time and date. A new event cannot be created on the same time as an existing event.",
42 "parameters": {
43 "type": "object",
44 "properties": {
45 "date": {
46 "type": "string",
47 "description": "the date on which the event starts, formatted as mm/dd/yy"
48 },
49 "time": {
50 "type": "string",
51 "description": "the time of the event, formatted using 24h military time formatting"
52 },
53 "duration": {
54 "type": "number",
55 "description": "the number of hours the event lasts for"
56 }
57 },
58 "required": ["date", "time", "duration"]
59 }
60 }
61 }
62]

Tool planning and calling

We can now run the tool use workflow. We can think of a tool use system as consisting of four components:

  • The user
  • The application
  • The LLM
  • The tools

At its most basic, these four components interact in a workflow through four steps:

  • Step 1: Get user message – The LLM gets the user message (via the application)
  • Step 2: Tool planning and calling – The LLM makes a decision on the tools to call (if any) and generates - the tool calls
  • Step 3: Tool execution - The application executes the tools and the results are sent to the LLM
  • Step 4: Response and citation generation – The LLM generates the response and citations to back to the user
PYTHON
1# Create custom system message
2system_message="""## Task and Context
3You are an assistant who assist new employees of Co1t with their first week. You respond to their questions and assist them with their needs. Today is Monday, June 24, 2024"""
4
5
6# Step 1: Get user message
7message = "Is there any message about getting setup with IT?"
8
9# Add the system and user messages to the chat history
10messages = [{"role": "system", "content": system_message},
11 {"role": "user", "content": message}]
12
13# Step 2: Tool planning and calling
14response = co.chat(
15 model="command-r-plus-08-2024",
16 messages=messages,
17 tools=tools
18 )
19
20if response.message.tool_calls:
21 print("Tool plan:")
22 print(response.message.tool_plan,"\n")
23 print("Tool calls:")
24 for tc in response.message.tool_calls:
25 print(f"Tool name: {tc.function.name} | Parameters: {tc.function.arguments}")
26
27 # Append tool calling details to the chat history
28 messages.append({"role": "assistant", "tool_calls": response.message.tool_calls, "tool_plan": response.message.tool_plan})
Tool plan:
I will search the user's emails for any messages about getting set up with IT.
Tool calls:
Tool name: search_emails | Parameters: {"query":"IT setup"}

Given three tools to choose from, the model is able to pick the right tool (in this case, search_emails) based on what the user is asking for.

Also, notice that the model first generates a plan about what it should do (“I will do …”) before actually generating the tool call(s).

Tool execution

PYTHON
1# Step 3: Tool execution
2for tc in response.message.tool_calls:
3 tool_result = functions_map[tc.function.name](**json.loads(tc.function.arguments))
4 tool_content = []
5 for data in tool_result:
6 tool_content.append({"type": "document", "document": {"data": json.dumps(data)}})
7 # Optional: add an "id" field in the "document" object, otherwise IDs are auto-generated
8 # Append tool results to the chat history
9 messages.append({"role": "tool", "tool_call_id": tc.id, "content": tool_content})
10
11 print("Tool results:")
12 for result in tool_content:
13 print(result)
Tool results:
{'type': 'document', 'document': {'data': '{"from": "it@co1t.com", "to": "david@co1t.com", "date": "2024-06-24", "subject": "Setting Up Your IT Needs", "text": "Greetings! To ensure a seamless start, please refer to the attached comprehensive guide, which will assist you in setting up all your work accounts."}'}}
{'type': 'document', 'document': {'data': '{"from": "john@co1t.com", "to": "david@co1t.com", "date": "2024-06-24", "subject": "First Week Check-In", "text": "Hello! I hope you\'re settling in well. Let\'s connect briefly tomorrow to discuss how your first week has been going. Also, make sure to join us for a welcoming lunch this Thursday at noon\\u2014it\'s a great opportunity to get to know your colleagues!"}'}}

Response and citation generation

PYTHON
1# Step 4: Response and citation generation
2response = co.chat(
3 model="command-r-plus-08-2024",
4 messages=messages,
5 tools=tools
6)
7
8# Append assistant response to the chat history
9messages.append({"role": "assistant", "content": response.message.content[0].text})
10
11# Print final response
12print("Response:")
13print(response.message.content[0].text)
14print("="*50)
15
16# Print citations (if any)
17if response.message.citations:
18 print("\nCITATIONS:")
19 for citation in response.message.citations:
20 print(citation, "\n")
Response:
Yes, there is an email from it@co1t.com with the subject 'Setting Up Your IT Needs'. It includes an attached guide to help you set up your work accounts.
==================================================
CITATIONS:
start=17 end=83 text="email from it@co1t.com with the subject 'Setting Up Your IT Needs'" sources=[ToolSource(type='tool', id='search_emails_wqs498sp2d07:0', tool_output={'date': '2024-06-24', 'from': 'it@co1t.com', 'subject': 'Setting Up Your IT Needs', 'text': 'Greetings! To ensure a seamless start, please refer to the attached comprehensive guide, which will assist you in setting up all your work accounts.', 'to': 'david@co1t.com'})]
start=100 end=153 text='attached guide to help you set up your work accounts.' sources=[ToolSource(type='tool', id='search_emails_wqs498sp2d07:0', tool_output={'date': '2024-06-24', 'from': 'it@co1t.com', 'subject': 'Setting Up Your IT Needs', 'text': 'Greetings! To ensure a seamless start, please refer to the attached comprehensive guide, which will assist you in setting up all your work accounts.', 'to': 'david@co1t.com'})]

Multi-step tool use

The model can execute more complex tasks in tool use – tasks that require tool calls to happen in a sequence. This is referred to as “multi-step” tool use.

Let’s create a function to called run_assistant to implement these steps, and along the way, print out the key events and messages. Optionally, this function also accepts the chat history as an argument to keep the state in a multi-turn conversation.

PYTHON
1model = "command-r-plus-08-2024"
2
3system_message="""## Task and Context
4You are an assistant who assists new employees of Co1t with their first week. You respond to their questions and assist them with their needs. Today is Monday, June 24, 2024"""
5
6def run_assistant(query, messages=None):
7 if messages is None:
8 messages = []
9
10 if "system" not in {m.get("role") for m in messages}:
11 messages.append({"role": "system", "content": system_message})
12
13 # Step 1: get user message
14 print(f"Question:\n{query}")
15 print("="*50)
16
17 messages.append({"role": "user", "content": query})
18
19 # Step 2: Generate tool calls (if any)
20 response = co.chat(
21 model=model,
22 messages=messages,
23 tools=tools
24 )
25
26 while response.message.tool_calls:
27
28 print("Tool plan:")
29 print(response.message.tool_plan,"\n")
30 print("Tool calls:")
31 for tc in response.message.tool_calls:
32 print(f"Tool name: {tc.function.name} | Parameters: {tc.function.arguments}")
33 print("="*50)
34
35 messages.append({"role": "assistant", "tool_calls": response.message.tool_calls, "tool_plan": response.message.tool_plan})
36
37 # Step 3: Get tool results
38 for idx, tc in enumerate(response.message.tool_calls):
39 tool_result = functions_map[tc.function.name](
40 **json.loads(tc.function.arguments)
41 )
42 tool_content = []
43 for data in tool_result:
44 tool_content.append({"type": "document", "document": {"data": json.dumps(data)}})
45 # Optional: add an "id" field in the "document" object, otherwise IDs are auto-generated
46 messages.append(
47 {"role": "tool", "tool_call_id": tc.id, "content": tool_content}
48 )
49
50 # Step 4: Generate response and citations
51 response = co.chat(
52 model=model,
53 messages=messages,
54 tools=tools
55 )
56
57 messages.append({"role": "assistant", "content": response.message.content[0].text})
58
59 # Print final response
60 print("Response:")
61 print(response.message.content[0].text)
62 print("="*50)
63
64 # Print citations (if any)
65 if response.message.citations:
66 print("\nCITATIONS:")
67 for citation in response.message.citations:
68 print(citation, "\n")
69
70 return messages

To illustrate the concept of multi-step tool user, let’s ask the assistant to block time for any lunch invites received in the email.

This requires tasks to happen over multiple steps in a sequence. Here, we see the assistant running these steps:

  • First, it calls the search_emails tool to find any lunch invites, which it found one.
  • Next, it calls the create_calendar_event tool to create an event to block the person’s calendar on the day mentioned by the email.

This is also an example of tool use enabling a write operation instead of just a read operation that we saw with RAG.

PYTHON
1messages = run_assistant("Can you check if there are any lunch invites, and for those days, create a one-hour event on my calendar at 12PM.")
Question:
Can you check if there are any lunch invites, and for those days, create a one-hour event on my calendar at 12PM.
==================================================
Tool plan:
I will first search the user's emails for lunch invites. Then, I will create a one-hour event on the user's calendar at 12PM for each day that the user has a lunch invite.
Tool calls:
Tool name: search_emails | Parameters: {"query":"lunch invites"}
==================================================
Tool plan:
I have found one lunch invite for Thursday at noon. I will now create a one-hour event on the user's calendar for Thursday at noon.
Tool calls:
Tool name: create_calendar_event | Parameters: {"date":"06/27/24","duration":1,"time":"12:00"}
==================================================
Response:
I found one lunch invite for Thursday, June 27, 2024. I have created a one-hour event on your calendar for that day at 12pm.
==================================================
CITATIONS:
start=29 end=53 text='Thursday, June 27, 2024.' sources=[ToolSource(type='tool', id='search_emails_1dxqzwragh9g:1', tool_output={'date': '2024-06-24', 'from': 'john@co1t.com', 'subject': 'First Week Check-In', 'text': "Hello! I hope you're settling in well. Let's connect briefly tomorrow to discuss how your first week has been going. Also, make sure to join us for a welcoming lunch this Thursday at noon—it's a great opportunity to get to know your colleagues!", 'to': 'david@co1t.com'})]
start=71 end=85 text='one-hour event' sources=[ToolSource(type='tool', id='create_calendar_event_w11caj6hmqz2:0', tool_output={'content': '"is_success"'})]
start=119 end=124 text='12pm.' sources=[ToolSource(type='tool', id='search_emails_1dxqzwragh9g:1', tool_output={'date': '2024-06-24', 'from': 'john@co1t.com', 'subject': 'First Week Check-In', 'text': "Hello! I hope you're settling in well. Let's connect briefly tomorrow to discuss how your first week has been going. Also, make sure to join us for a welcoming lunch this Thursday at noon—it's a great opportunity to get to know your colleagues!", 'to': 'david@co1t.com'})]

In this tutorial, you learned about:

  • How to create tools
  • How tool planning and calling happens
  • How tool execution happens
  • How to generate the response and citations
  • How to run tool use in a multi-step scenario

And that concludes our 7-part Cohere tutorial. We hope that they have provided you with a foundational understanding of the Cohere API, the available models and endpoints, and the types of use cases that you can build with them.

To continue your learning, check out: