Building an Agent with Cohere
Tool use extends the ideas from RAG, where external systems are used to guide the response of an LLM, but by leveraging a much bigger set of tools than what’s possible with RAG. The concept of tool use leverages LLMs’ useful feature of being able to act as a reasoning and decision-making engine.
While RAG enables applications that can answer questions, tool use enables those that can automate tasks.
Tool use also enables developers to build agentic applications that can take actions, that is, doing both read and write operations on an external system.
In this tutorial, you’ll learn about:
- Creating tools
- Tool planning and calling
- Tool execution
- Response and citation generation
- Multi-step tool use
You’ll learn these by building an onboarding assistant for new hires.
Setup
To get started, first we need to install the cohere
library and create a Cohere client.
Creating tools
The pre-requisite, before we can run a tool use workflow, is to set up the tools. Let’s create three tools:
search_faqs
: A tool for searching the FAQs. For simplicity, we’ll not implement any retrieval logic, but we’ll simply pass a list of pre-defined documents, which are the FAQ documents we had used in the text embeddings section.search_emails
: A tool for searching the emails. Same as above, we’ll simply pass a list of pre-defined emails from the Reranking section.create_calendar_event
: A tool for creating new calendar events. Again, for simplicity, we’ll not implement actual event bookings, but will return a mock success event. In practice, we can connect to a calendar service API and implement all the necessary logic here.
Here, we are defining a Python function for each tool, but more broadly, the tool can be any function or service that can receive and send objects.
The second and final setup step is to define the tool schemas in a format that can be passed to the Chat endpoint. The schema must contain the following fields: name
, description
, and parameter_definitions
in the format shown below.
This schema informs the LLM about what the tool does, and the LLM decides whether to use a particular tool based on it. Therefore, the more descriptive and specific the schema, the more likely the LLM will make the right tool call decisions.
Tool planning and calling
We can now run the tool use workflow. We can think of a tool use system as consisting of four components:
- The user
- The application
- The LLM
- The tools
At its most basic, these four components interact in a workflow through four steps:
- Step 1: Get user message – The LLM gets the user message (via the application)
- Step 2: Tool planning and calling – The LLM makes a decision on the tools to call (if any) and generates - the tool calls
- Step 3: Tool execution - The application executes the tools and the results are sent to the LLM
- Step 4: Response and citation generation – The LLM generates the response and citations to back to the user
Given three tools to choose from, the model is able to pick the right tool (in this case, search_emails
) based on what the user is asking for.
Also, notice that the model first generates a plan about what it should do (“I will do …”) before actually generating the tool call(s).
Tool execution
Response and citation generation
Multi-step tool use
The model can execute more complex tasks in tool use – tasks that require tool calls to happen in a sequence. This is referred to as “multi-step” tool use.
Let’s create a function to called run_assistant
to implement these steps, and along the way, print out the key events and messages. Optionally, this function also accepts the chat history as an argument to keep the state in a multi-turn conversation.
To illustrate the concept of multi-step tool user, let’s ask the assistant to block time for any lunch invites received in the email.
This requires tasks to happen over multiple steps in a sequence. Here, we see the assistant running these steps:
- First, it calls the
search_emails
tool to find any lunch invites, which it found one. - Next, it calls the
create_calendar_event
tool to create an event to block the person’s calendar on the day mentioned by the email.
This is also an example of tool use enabling a write operation instead of just a read operation that we saw with RAG.
In this tutorial, you learned about:
- How to create tools
- How tool planning and calling happens
- How tool execution happens
- How to generate the response and citations
- How to run tool use in a multi-step scenario
And that concludes our 7-part Cohere tutorial. We hope that they have provided you with a foundational understanding of the Cohere API, the available models and endpoints, and the types of use cases that you can build with them.
To continue your learning, check out: