Building an Agent with Cohere

Open in Colab

Tool use extends the ideas from RAG, where external systems are used to guide the response of an LLM, but by leveraging a much bigger set of tools than what’s possible with RAG. The concept of tool use leverages LLMs’ useful feature of being able to act as a reasoning and decision-making engine.

While RAG enables applications that can answer questions, tool use enables those that can automate tasks.

Tool use also enables developers to build agentic applications that can take actions, that is, doing both read and write operations on an external system.

In this tutorial, you’ll learn about:

  • Creating tools
  • Tool planning and calling
  • Tool execution
  • Response and citation generation
  • Multi-step tool use

You’ll learn these by building an onboarding assistant for new hires.

Setup

To get started, first we need to install the cohere library and create a Cohere client.

PYTHON
1# pip install cohere numpy
2
3import numpy as np
4import cohere
5
6co = cohere.Client("COHERE_API_KEY") # Get your API key: https://dashboard.cohere.com/api-keys

Creating tools

The pre-requisite, before we can run a tool use workflow, is to set up the tools. Let’s create three tools:

  • search_faqs: A tool for searching the FAQs. For simplicity, we’ll not implement any retrieval logic, but we’ll simply pass a list of pre-defined documents, which are the FAQ documents we had used in the text embeddings section.
  • search_emails: A tool for searching the emails. Same as above, we’ll simply pass a list of pre-defined emails from the Reranking section.
  • create_calendar_event: A tool for creating new calendar events. Again, for simplicity, we’ll not implement actual event bookings, but will return a mock success event. In practice, we can connect to a calendar service API and implement all the necessary logic here.

Here, we are defining a Python function for each tool, but more broadly, the tool can be any function or service that can receive and send objects.

PYTHON
1# Create the tools
2def search_faqs(query):
3 faqs = [
4 {"text": "Reimbursing Travel Expenses: Easily manage your travel expenses by submitting them through our finance tool. Approvals are prompt and straightforward."},
5 {"text": "Working from Abroad: Working remotely from another country is possible. Simply coordinate with your manager and ensure your availability during core hours."}
6 ]
7 return {"faqs" : faqs}
8
9def search_emails(query):
10 emails = [
11 {"from": "it@co1t.com", "to": "david@co1t.com", "date": "2024-06-24", "subject": "Setting Up Your IT Needs", "text": "Greetings! To ensure a seamless start, please refer to the attached comprehensive guide, which will assist you in setting up all your work accounts."},
12 {"from": "john@co1t.com", "to": "david@co1t.com", "date": "2024-06-24", "subject": "First Week Check-In", "text": "Hello! I hope you're settling in well. Let's connect briefly tomorrow to discuss how your first week has been going. Also, make sure to join us for a welcoming lunch this Thursday at noon—it's a great opportunity to get to know your colleagues!"}
13 ]
14 return {"emails" : emails}
15
16def create_calendar_event(date: str, time: str, duration: int):
17 # You can implement any logic here
18 return {"is_success": True,
19 "message": f"Created a {duration} hour long event at {time} on {date}"}
20
21functions_map = {
22 "search_faqs": search_faqs,
23 "search_emails": search_emails,
24 "create_calendar_event": create_calendar_event
25}

The second and final setup step is to define the tool schemas in a format that can be passed to the Chat endpoint. The schema must contain the following fields: name, description, and parameter_definitions in the format shown below.

This schema informs the LLM about what the tool does, and the LLM decides whether to use a particular tool based on it. Therefore, the more descriptive and specific the schema, the more likely the LLM will make the right tool call decisions.

PYTHON
1# Define the tools
2tools = [
3 {
4 "name": "search_faqs",
5 "description": "Given a user query, searches a company's frequently asked questions (FAQs) list and returns the most relevant matches to the query.",
6 "parameter_definitions": {
7 "query": {
8 "description": "The query from the user",
9 "type": "str",
10 "required": True
11 }
12 }
13 },
14 {
15 "name": "search_emails",
16 "description": "Given a user query, searches a person's emails and returns the most relevant matches to the query.",
17 "parameter_definitions": {
18 "query": {
19 "description": "The query from the user",
20 "type": "str",
21 "required": True
22 }
23 }
24 },
25 {
26 "name": "create_calendar_event",
27 "description": "Creates a new calendar event of the specified duration at the specified time and date. A new event cannot be created on the same time as an existing event.",
28 "parameter_definitions": {
29 "date": {
30 "description": "the date on which the event starts, formatted as mm/dd/yy",
31 "type": "str",
32 "required": True
33 },
34 "time": {
35 "description": "the time of the event, formatted using 24h military time formatting",
36 "type": "str",
37 "required": True
38 },
39 "duration": {
40 "description": "the number of hours the event lasts for",
41 "type": "float",
42 "required": True
43 }
44 }
45 }
46]

Tool planning and calling

We can now run the tool use workflow. We can think of a tool use system as consisting of four components:

  • The user
  • The application
  • The LLM
  • The tools

At its most basic, these four components interact in a workflow through four steps:

  • Step 1: Get user message – The LLM gets the user message (via the application)
  • Step 2: Tool planning and calling – The LLM makes a decision on the tools to call (if any) and generates - the tool calls
  • Step 3: Tool execution - The application executes the tools and the results are sent to the LLM
  • Step 4: Response and citation generation – The LLM generates the response and citations to back to the user
PYTHON
1# Step 1: Get user message
2message = "Any messages about getting setup with IT?"
3
4preamble="""## Task & Context
5You are an assistant who assist new employees of Co1t with their first week. You respond to their questions and assist them with their needs. Today is Monday, June 24, 2024"""
6
7# Step 2: Tool planning and calling
8response = co.chat(
9 message=message,
10 preamble=preamble,
11 tools=tools)
12
13if response.tool_calls:
14 print("Tool plan:")
15 print(response.text,"\n")
16
17 print("Tool calls:")
18 for call in response.tool_calls:
19 print(f"Tool name: {call.name} | Parameters: {call.parameters}")
Tool plan:
I will search the user's emails for any messages about getting set up with IT.
Tool calls:
Tool name: search_emails | Parameters: {'query': 'IT setup'}

Given three tools to choose from, the model is able to pick the right tool (in this case, search_emails) based on what the user is asking for.

Also, notice that the model first generates a plan about what it should do (“I will do …”) before actually generating the tool call(s).

Tool execution

PYTHON
1# Step 3: Tool execution
2tool_results = []
3for tc in response.tool_calls:
4 tool_call = {"name": tc.name, "parameters": tc.parameters}
5 tool_output = functions_map[tc.name](**tc.parameters)
6 tool_results.append({"call": tool_call, "outputs": [tool_output]})
7
8print("Tool results:")
9for result in tool_results:
10 print(result)
Tool results:
{'call': {'name': 'search_emails', 'parameters': {'query': 'IT setup'}}, 'outputs': [{'emails': [{'from': 'it@co1t.com', 'to': 'david@co1t.com', 'date': '2024-06-24', 'subject': 'Setting Up Your IT Needs', 'text': 'Greetings! To ensure a seamless start, please refer to the attached comprehensive guide, which will assist you in setting up all your work accounts.'}, {'from': 'john@co1t.com', 'to': 'david@co1t.com', 'date': '2024-06-24', 'subject': 'First Week Check-In', 'text': "Hello! I hope you're settling in well. Let's connect briefly tomorrow to discuss how your first week has been going. Also, make sure to join us for a welcoming lunch this Thursday at noon—it's a great opportunity to get to know your colleagues!"}]}]}

Response and citation generation

PYTHON
1# Step 4: Response and citation generation
2response = co.chat(
3 message="", # In response generation, we set the message as empty
4 preamble=preamble,
5 tools=tools,
6 tool_results=tool_results,
7 chat_history=response.chat_history
8)
9
10# Print final response
11print("Final response:")
12print(response.text)
13print("="*50)
14
15# Print citations (if any)
16if response.citations:
17 print("\nCITATIONS:")
18 for citation in response.citations:
19 print(citation)
20
21 print("\nCITED REFERENCES:")
22 for document in response.documents:
23 print(document)
Final response:
You have an email from IT with a comprehensive guide attached to help you set up your work accounts.
==================================================
CITATIONS:
start=12 end=25 text='email from IT' document_ids=['search_emails:0:2:0']
start=33 end=61 text='comprehensive guide attached' document_ids=['search_emails:0:2:0']
start=74 end=99 text='set up your work accounts' document_ids=['search_emails:0:2:0']
CITED REFERENCES:
{'emails': '[{"date":"2024-06-24","from":"it@co1t.com","subject":"Setting Up Your IT Needs","text":"Greetings! To ensure a seamless start, please refer to the attached comprehensive guide, which will assist you in setting up all your work accounts.","to":"david@co1t.com"},{"date":"2024-06-24","from":"john@co1t.com","subject":"First Week Check-In","text":"Hello! I hope you\'re settling in well. Let\'s connect briefly tomorrow to discuss how your first week has been going. Also, make sure to join us for a welcoming lunch this Thursday at noon—it\'s a great opportunity to get to know your colleagues!","to":"david@co1t.com"}]', 'id': 'search_emails:0:2:0', 'tool_name': 'search_emails'}

Multi-step tool use

The model can execute more complex tasks in tool use – tasks that require tool calls to happen in a sequence. This is referred to as “multi-step” tool use.

Let’s create a function to called run_assistant to implement these steps, and along the way, print out the key events and messages. Optionally, this function also accepts the chat history as an argument to keep the state in a multi-turn conversation.

PYTHON
1model = "command-r-plus-08-2024"
2
3preamble="""## Task & Context
4You are an assistant who assists new employees of Co1t with their first week. You respond to their questions and assist them with their needs. Today is Monday, June 24, 2024"""
5
6# A function that runs multi-step tool use
7def run_assistant(message, chat_history=[]):
8 # Step 1: get user message
9 print(f"Question:\n{message}")
10 print("="*50)
11
12 # Step 2: Generate tool calls (if any)
13 response = co.chat(
14 message=message,
15 model=model,
16 preamble=preamble,
17 tools=tools,
18 chat_history=chat_history
19 )
20
21 # Tool execution loop
22 while response.tool_calls:
23 tool_calls = response.tool_calls
24
25 if response.text:
26 print("Intermediate response:")
27 print(response.text,"\n")
28 print("Tool calls:")
29 for call in tool_calls:
30 print(f"Tool name: {call.name} | Parameters: {call.parameters}")
31 print("="*50)
32
33 # Step 3: Get tool results
34 tool_results = []
35 for tc in tool_calls:
36 tool_call = {"name": tc.name, "parameters": tc.parameters}
37 tool_output = functions_map[tc.name](**tc.parameters)
38 tool_results.append({"call": tool_call, "outputs": [tool_output]})
39
40 # Step 4: Generate response and citations
41 response = co.chat(
42 message="",
43 model=model,
44 preamble=preamble,
45 tools=tools,
46 tool_results=tool_results,
47 chat_history=response.chat_history
48 )
49
50 chat_history = response.chat_history
51
52 # Print final response
53 print("Final response:")
54 print(response.text)
55 print("="*50)
56
57 # Print citations (if any)
58 if response.citations:
59 print("\nCITATIONS:")
60 for citation in response.citations:
61 print(citation)
62
63 print("\nCITED REFERENCES:")
64 for document in response.documents:
65 print(document)
66
67 return chat_history

To illustrate the concept of multi-step tool user, let’s ask the assistant to block time for any lunch invites received in the email.

This requires tasks to happen over multiple steps in a sequence. Here, we see the assistant running these steps:

  • First, it calls the search_emails tool to find any lunch invites, which it found one.
  • Next, it calls the create_calendar_event tool to create an event to block the person’s calendar on the day mentioned by the email.

This is also an example of tool use enabling a write operation instead of just a read operation that we saw with RAG.

PYTHON
1chat_history = run_assistant("Can you check if there are any lunch invites, and for those days, block an hour on my calendar from 12-1PM.")
Question:
Can you check if there are any lunch invites, and for those days, block an hour on my calendar from 12-1PM.
==================================================
Intermediate response:
I will search the user's emails for lunch invites, and then create calendar events for the dates and times of those invites.
Tool calls:
Tool name: search_emails | Parameters: {'query': 'lunch invite'}
==================================================
Intermediate response:
I have found one lunch invite for Thursday 27 June at noon. I will now create a calendar event for this.
Tool calls:
Tool name: create_calendar_event | Parameters: {'date': '06/27/24', 'duration': 1, 'time': '12:00'}
==================================================
Final response:
I found one lunch invite for Thursday 27 June at noon. I have created a calendar event for this.
==================================================
CITATIONS:
start=29 end=53 text='Thursday 27 June at noon' document_ids=['search_emails:0:2:0']
start=62 end=95 text='created a calendar event for this' document_ids=['create_calendar_event:0:4:0']
CITED REFERENCES:
{'emails': '[{"date":"2024-06-24","from":"it@co1t.com","subject":"Setting Up Your IT Needs","text":"Greetings! To ensure a seamless start, please refer to the attached comprehensive guide, which will assist you in setting up all your work accounts.","to":"david@co1t.com"},{"date":"2024-06-24","from":"john@co1t.com","subject":"First Week Check-In","text":"Hello! I hope you\'re settling in well. Let\'s connect briefly tomorrow to discuss how your first week has been going. Also, make sure to join us for a welcoming lunch this Thursday at noon—it\'s a great opportunity to get to know your colleagues!","to":"david@co1t.com"}]', 'id': 'search_emails:0:2:0', 'tool_name': 'search_emails'}
{'id': 'create_calendar_event:0:4:0', 'is_success': 'true', 'message': 'Created a 1 hour long event at 12:00 on 06/27/24', 'tool_name': 'create_calendar_event'}

In this tutorial, you learned about:

  • How to create tools
  • How tool planning and calling happens
  • How tool execution happens
  • How to generate the response and citations
  • How to run tool use in a multi-step scenario

And that concludes our 7-part Cohere tutorial. We hope that they have provided you with a foundational understanding of the Cohere API, the available models and endpoints, and the types of use cases that you can build with them.

To continue your learning, check out: