Multi-step Tool Use (Agents)

Tool use is a technique which allows Cohere’s models to invoke external tools: search engines, APIs, functions, databases, and so on. Given a list of tool definitions, the model will generate a plan of action and decide which tools to use, in which order, and with what parameters.

For example, given the web-search tool, the model can start answering complex questions that require performing internet searches.

Also, note that multi-step is enabled by default.

Using the Chat API with Tools

Step 1: Define the tools

1
# define the `web_search` tool.
2
3
def web_search(query: str) -> list[dict]:
4
  # your code for performing a web search goes here
5
  # return [{
6
  #		"url": "https://en.wikipedia.org/wiki/Ontario",
7
  #		"text": "The capital of Ontario is Toronto, ..."
8
  #	}]
9
10
web_search_tool = {
11
  "name": "web_search",
12
  "description": "performs a web search with the specified query",
13
  "parameter_definitions": {
14
    "query": {
15
      "description": "the query to look up",
16
      "type": "str",
17
      "required": True
18
    }
19
  }
20
}

Step 2: Ask model for tool calls and send back tool results

1
import cohere
2
co = cohere.Client(api_key="<YOUR API KEY>")
3
4
message = "Who is the mayor of the capital of Ontario?"
5
model = "command-r-plus-08-2024"
6
7
# STEP 2: Check what tools the model wants to use and how
8
9
res = co.chat(model=model, message=message, force_single_step=False, tools=[web_search_tool])
10
11
# as long as the model sends back tool_calls,
12
# keep invoking tools and sending the results back to the model
13
while res.tool_calls:
14
  print(res.text) # This will be an observation and a plan with next steps
15
  tool_results = []
16
  for call in res.tool_calls:
17
    # use the `web_search` tool with the search query the model sent back
18
    web_search_results = {"call": call, "outputs": web_search(call.parameters["query"])}
19
    tool_results.append(web_search_results)
20
    
21
  # call chat again with tool results
22
  res = co.chat(
23
    model="command-r-plus-08-2024",
24
    chat_history=res.chat_history,
25
    message="",
26
    force_single_step=False,
27
    tools=[web_search_tool],
28
    tool_results=tool_results
29
  )
30
  
31
 print(res.text) # "The mayor of Toronto, the capital of Ontario is Olivia Chow"

How Does Multi-step Tool Use Work?

Here’s an outline of the basic steps involved in multi-step tool use:

• Given a user request, the model comes up with a plan to solve the problem which answers questions such as “Which tools should be used,” and “In what order should they be used.”
• The model then carries out the plan by repeatedly executing actions (using whatever tools are appropriate), reasoning over the results, and re-evaluating the plan.
• After each Action -> Observation ->Reflection cycle, the model reflects about what to do next. This reflection involves analyzing what has been figured out so far, determining whether any changes need to be made to the plan, and what to do next. The model can take as many steps as it deems necessary.
• Once the model decides it knows how to answer the user question, it proceeds to generating the final response.

How Does Multi-step Tool Use Differ From Single-step Tool Use?

In single-step tool use, the model is also equipped with access to a bevy of tools to answer a question.

That said single-step tool use only facilitates the model calling multiple tools during a single step. The model cannot execute a sequence of steps, and it cannot use the results from one tool call in a subsequent step.

FAQs

When Should I Use Multi-step Tool Use?

For more complex queries it’s probably better to operate in multi-step mode, which is the default (you can operate in single-step mode by setting force_single_step=True). In multi-step mode, the model can reason across steps and select multiple tools to answer a question completely.

What is the difference between tool use and Retrieval Augmented Generation (RAG)?

Tool use is a natural extension of retrieval augmented generation (RAG). RAG is about enabling the model to interact with an information retrieval system (like a vector database). Our models are trained to be excellent at RAG use cases.

Tool use pushes this further, allowing Cohere models to go far beyond information retrieval, interact with search engines, APIs, functions, databases, and many other tools.