Implementing a Multi-Step Agent with Langchain

In this document, we’ll go through the nuts-and-bolts of building a generative-AI agent with Cohere’s multi-step tool use functionality and the Langchain framework.

Building the Langchain ReAct Agent

Multi-step tool use with Cohere can be implemented using the Langchain framework, which conveniently comes with many pre-defined tools. More specifically, we recommend using the ReAct agent abstraction in Langchain, powered by create_cohere_react_agent. Let’s see how we can easily build an agent, using the multi-step tool use capabilities of Langchain and Cohere.

Jupyter Notebook

The example below is also available in this Jupyter Notebook for convenience.

First, we’ll install the dependencies. (Note: the ! is required for notebooks, but you must omit it if you’re in the command line).

PYTHON
1! pip install --quiet langchain langchain_cohere langchain_experimental

Second, we define some tools to equip your agent. Langchain comes out-of-the-box with more than 50 predefined tools, including web search, a python interpreter, vector stores, and many others.

Below, we’ve included two code snippets, equipping the agent with the Web Search and Python interpreter tools, respectively.

Example: define the Web Search tool

PYTHON
1from langchain_community.tools.tavily_search import TavilySearchResults
2
3os.environ["TAVILY_API_KEY"] = #<your-api-key>
4
5internet_search = TavilySearchResults()
6internet_search.name = "internet_search"
7internet_search.description = "Returns a list of relevant document snippets for a textual query retrieved from the internet."
8
9
10from langchain_core.pydantic_v1 import BaseModel, Field
11class TavilySearchInput(BaseModel):
12 query: str = Field(description="Query to search the internet with")
13internet_search.args_schema = TavilySearchInput

Example: define the Python Interpreter tool

PYTHON
1from langchain.agents import Tool
2from langchain_experimental.utilities import PythonREPL
3
4python_repl = PythonREPL()
5python_tool = Tool(
6 name="python_repl",
7 description="Executes python code and returns the result. The code runs in astatic sandbox without interactive mode, so print output or save output to a file.",
8 func=python_repl.run,
9)
10python_tool.name = "python_interpreter"
11
12# from langchain_core.pydantic_v1 import BaseModel, Field
13class ToolInput(BaseModel):
14 code: str = Field(description="Python code to execute.")
15python_tool.args_schema = ToolInput

Even better any Python function can easily be transformed into a Langchain tool by using the @tool decorator. As a best practice, should specify the tool name, definition, and arguments schema.

Example: define a custom tool

PYTHON
1from langchain_core.tools import tool
2import random
3
4@tool
5def random_operation_tool(a: int, b: int):
6 """Calculates a random operation between the inputs."""
7 coin_toss = random.uniform(0, 1)
8 if coin_toss > 0.5:
9 return {'output': a*b}
10 else:
11 return {'output': a+b}
12
13random_operation_tool.name = "random_operation" # use python case
14random_operation_tool.description = "Calculates a random operation between the inputs."
15
16from langchain_core.pydantic_v1 import BaseModel, Field
17class random_operation_inputs(BaseModel):
18 a: int = Field(description="First input")
19 b: int = Field(description="Second input")
20random_operation_tool.args_schema = random_operation_inputs

Third, create a ReAct agent in Langchain. The model can dynamically pick the right tool(s) for the user query, call them in a sequence, analyze the results, and self-reflect. Note that your ReAct agent can optionally take an input preamble.

PYTHON
1from langchain.agents import AgentExecutor
2from langchain_cohere.react_multi_hop.agent import create_cohere_react_agent
3from langchain_core.prompts import ChatPromptTemplate
4from langchain_cohere.chat_models import ChatCohere
5
6# LLM
7llm = ChatCohere(model="command-r-plus-08-2024", temperature=0.3)
8
9# Preamble
10preamble = """
11You are an expert who answers the user's question with the most relevant datasource.
12You are equipped with an internet search tool and a special vectorstore of information
13about how to write good essays.
14"""
15
16# Prompt template
17prompt = ChatPromptTemplate.from_template("{input}")
18
19# Create the ReAct agent
20agent = create_cohere_react_agent(
21 llm=llm,
22 tools=[internet_search, vectorstore_search, python_tool],
23 prompt=prompt,
24)
25
26agent_executor = AgentExecutor(agent=agent,
27 tools=[internet_search, vectorstore_search, python_tool],
28 verbose=True)

Finally, call your agent with a question!

PYTHON
1agent_executor.invoke({
2 "input": "I want to write an essay about the Roman Empire. Any tips for writing an essay? Any fun facts?",
3 "preamble": preamble,
4})

Inspecting the Logs

We can get some insight into what’s going on under the hood by taking a look at the logs (we’ve added # comments throughout for context):

ASP.NET
1> Entering new AgentExecutor chain...
2
3
4# Here is the model plan
5I will search for tips on writing an essay and fun facts about the Roman Empire.
6
7
8# The model decides to use a first tool: the vector store
9{'tool_name': 'vectorstore_search', 'parameters': {'query': 'tips for writing an essay'}}
10
11# Here are the results from the vector store call: retrieved passages
12I should have asked how do you write essays well? Though
13these seem only phrasing apart, their answers diverge. [ … more of retrieved snippet 1 … ]
14
15didn't have edge with any of them. To start writing an essay, you
16need [ … more of retrieved snippet 2 … ]
17
18You don't have to get an answer right the first time, but there's
19no excuse for not getting it right eventually, because [ more of retrieved snippet 3 … ]
20
21
22# The model decides to use another tool: web search
23{'tool_name': 'internet_search', 'parameters': {'query': 'fun facts about the roman empire'}}
24
25# Here are the results from the web search call: retrieved passages
26[{'url': 'https://www.natgeokids.com/uk/discover/history/romans/10-facts-about-the-ancient-romans/', 'content': 'i love this website\nBIG BOBBY\nbooby\nI love shell my bae;)\ni like bobby fishes ;0\nI like turtles\nOmg soy cool\ngreeeeeeeeeeeeaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaatttttttttttttttttttttttt\nbest fact ever\nthis artical is cool\nHANDY\nrubbish did not help what so ever\nha\nRocking\nTHIS IS THE BEST\nproper rad in it cool\nthis is cool\nawesomeness\nawsome\nawsome\nthank you captain\nit is a lot of help\ni like this\nwebsite it helps me on my projects and isabel likes munier\nmark uses this for research\nlot of help\nthis is awsome\nTHE BEST BOOBOO\nCool webpage helped me get 4 housepoints\n This helped me A LOT on a school project\ncool wow awesomoe\nCOOL WEBSITE LOL\nthis helped me with a school project :)\nthat was awesome\ncool\nthat helped me out for my research test\nReally its very cool really COOL\nLIKE COOL best website so far its nice\nI love it\nnice facts\nIt help with my history\n i mean u made animaljam a awesome nice safe place for kids and this site to have kids a safe website to get facts for reports and stuff\nLots of Love ,\nRose\npretty good website if u ask me\nbut definently not gonna use it on a daily basis\nIll try it again another time\ngood\nCool webcite\nterrible\nquite impressive\nAwesome website it real helps\nits good\nthis is a great website! You really a lot with my project!:)\nthis has helleped\nme get\nmy progect\ndone\nthank you\nsoooooooooooooooooo\nmuchchchchchch\nthis helleped me\nsooooooooo much with my progect thank you\nvery good website\nthank us very much your nice one today!!\n'}, {'url': 'https://ohfact.com/roman-empire-facts/', 'content': 'Learn about the ancient Roman Civilization, its history, culture, army, architecture, food and more from this list of 27 facts. Discover how the Romans started, conquered, lived, died and influenced the world with their legends, myths and facts.'}, {'url': 'https://factnight.com/fun-facts-about-the-roman-empire/', 'content': 'The Roman Empire was one of the most influential and significant civilizations in world history. At its peak, the empire stretched from North Africa to Britain, reigning over 60 million people. From its legendary beginnings and remarkable achievements to its eventual decline and fall, the Roman Empire is a fascinating topic full of little-known facts and intriguing trivia.'}, {'url': 'https://www.historyhit.com/facts-about-ancient-rome-and-the-romans/', 'content': 'The Enduring Legacy of C.S. Lewis\nMargaret J. Winkler: A Forgotten Pioneer in Disney’s Success\n10 Facts About Harper Lee\nAntarctica Expedition Cruise\nUncover Pompeii\nSophie Hay and Tristan Hughes\nRediscovering Richard III with Matt Lewis\nOrder the History Hit Miscellany\nHistory Hit Holidays\nGift Subscriptions\n100 Facts About Ancient Rome and the Romans\nRome wasn’t built in a day, as the cliché reminds us. The Crossing of the Rhine in 405/6 AD brought around 100,000 barbarians into the Empire\nBarbarian factions, tribes and war leaders were now a factor in the power struggles at the top of Roman politics and one of the once-strong boundaries of the Empire had proved to be permeable.\n Related Articles\n10 Facts About Saint Andrew\nThe Rise of Pompey the Great, the ‘Roman Alexander’\nWatch and Listen\nCleopatra\nSex in Ancient Rome\nRelated Locations\nBaelo Claudia\nMausoleum of Cecilia Metella\nColin Ricketts\n30 July 2021\n By the fourth century BC, the story was accepted by Romans who were proud of their warrior founder\nThe story was included in the first history of the city, by the Greek writer Diocles of Peparethus, and the twins and their wolf step-mother were depicted on Rome’s first coins.\n The History Hit Miscellany of Facts, Figures and Fascinating Finds\nA History of England: Part One\nDragons: Myth & Reality\nA Tudor Wonder - Hardwick Hall\nThe Battle of Shrewsbury\nEurope’s 1848 Revolutions\nThe Boston Tea Party\nHow Did 3 People Seemingly Escape From Alcatraz?\n'}, {'url': 'https://www.countryfaq.com/facts-about-the-roman-empire/', 'content': 'Facts about the Roman Empire. Explore some of the interesting, fun, cool facts bout the Roman Empire: 1. The Magnificent Roman Empire. The Roman Empire, a colossal entity of unparalleled grandeur, occupies an indomitable position within the annals of human history, a name that resonates resoundingly across the eons.'}]Relevant Documents: 0,3,4,5
27
28
29# The model decides it has enough info to generate a final response.
30
31# Below is the answer by the model
32Answer: Here are some tips for writing an essay:
33- Start with a question that spurs some response.
34- Don't choose a topic at random, make sure you have a way in, a new insight or approach.
35- You don't need a complete thesis, just a gap to explore.
36- You can get ideas by talking to people, reading, doing and building things, and going places and seeing things.
37- You can improve the quality of your ideas by increasing the breadth and depth of what goes in.
38- You can get breadth by reading and talking about a wide range of topics.
39- You can get depth by doing and having to solve problems.
40- You can also get ideas by talking to people who make you have new ideas.
41
42Here are some fun facts about the Roman Empire:
43- At its peak, the empire stretched from North Africa to Britain, reigning over 60 million people.
44- The story of Rome's warrior founder and the twins and their wolf step-mother was depicted on Rome's first coins.
45- The Crossing of the Rhine in 405/6 AD brought around 100,000 barbarians into the Empire.
46
47# Below is the answer by the model, with citations!
48Cited Documents: 0,3,4,5
49Grounded answer: Here are some tips for writing an essay:
50- <co: 0>Start with a question that spurs some response</co: 0>.
51- <co: 0>Don't choose a topic at random, make sure you have a way in, a new insight or approach</co: 0>.
52- <co: 0>You don't need a complete thesis, just a gap to explore</co: 0>.
53- <co: 0>You can get ideas by talking to people, reading, doing and building things, and going places and seeing things</co: 0>.
54- <co: 0>You can improve the quality of your ideas by increasing the breadth and depth of what goes in</co: 0>.
55- <co: 0>You can get breadth by reading and talking about a wide range of topics</co: 0>.
56- <co: 0>You can get depth by doing and having to solve problems</co: 0>.
57- <co: 0>You can also get ideas by talking to people who make you have new ideas</co: 0>.
58
59Here are some fun facts about the Roman Empire:
60- <co: 3>At its peak, the empire stretched from North Africa to Britain, reigning over 60 million people</co: 3>.
61- <co: 4>The story of Rome's warrior founder and the twins and their wolf step-mother was depicted on Rome's first coins</co: 4>.
62- <co: 4>The Crossing of the Rhine in 405/6 AD brought around 100,000 barbarians into the Empire</co: 4>.
63
64> Finished chain.

Some Useful Tools

Beyond the web search tool and the Python interpreter tool shared in the code snippets above, we have found some tools to be particularly useful. Here’s an example of leveraging a vector store for greater functionality:

PYTHON
1# You can easily equip your agent with a vector store!
2
3from langchain.text_splitter import RecursiveCharacterTextSplitter
4from langchain_community.document_loaders import WebBaseLoader
5from langchain_community.vectorstores import FAISS
6from langchain_cohere import CohereEmbeddings
7
8# Set embeddings
9embd = CohereEmbeddings()
10
11# Docs to index
12urls = [
13 "https://paulgraham.com/best.html",
14]
15
16# Load
17docs = [WebBaseLoader(url).load() for url in urls]
18docs_list = [item for sublist in docs for item in sublist]
19
20# Split
21text_splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(
22 chunk_size=512, chunk_overlap=0
23)
24doc_splits = text_splitter.split_documents(docs_list)
25
26# Add to vectorstore
27vectorstore = FAISS.from_documents(
28 documents=doc_splits,
29 embedding=embd,
30)
31
32vectorstore_retriever = vectorstore.as_retriever()
33
34
35from langchain.tools.retriever import create_retriever_tool
36
37vectorstore_search = create_retriever_tool(
38 retriever=vectorstore_retriever,
39 name="vectorstore_search",
40 description="Retrieve relevant info from a vectorstore that contains information from Paul Graham about how to write good essays."
41)

Multi-turn Conversations and Chat History

So far, we asked one-off questions to the ReAct agent. In many enterprise applications, end users want to have conversations with the ReAct agent.

The ReAct agent can handle multi-turn conversations by using chat_history.

PYTHON
1# Step 1: Construct the chat history as a list of LangChain Messages, ending with the last user message
2from langchain_core.messages import HumanMessage, AIMessage
3
4chat_history = [
5 HumanMessage(content="I'm considering switching to Oracle for my CRM."),
6 AIMessage(content="That sounds like a good idea! How can I help you?"),
7 HumanMessage(content="Recap all the info you can find about their offering."),
8]
9
10prompt = ChatPromptTemplate.from_messages(chat_history)
11
12# Step 2: When you make the agent, specify the chat_history as the prompt
13agent = create_cohere_react_agent(
14 llm=llm,
15 tools=[internet_search, vectorstore_search, python_tool],
16 prompt=prompt,
17)
18
19agent_executor = AgentExecutor(agent=agent,
20 tools=[internet_search, vectorstore_search, python_tool],
21 verbose=True)
22
23# Step 3: When you invoke the agent_executor there's no need to pass anything else into invoke
24response = agent_executor.invoke({
25 "preamble": preamble,
26})
27
28response['output']

Can the ReAct Agent Directly Answer a Question?

Yes. The ReAct agent from Cohere comes out of the box with the ability to answer a user question directly. This happens when answering the user’s question doesn’t require using a tool.

For example, let’s look at the following question:

PYTHON
1agent_executor.invoke({
2 "input": "Hey how are you?",
3})

By inspecting the logs, we see that the ReAct agent decided to just respond directly.

1> Entering new AgentExecutor chain...
2Plan: I will respond to the user's greeting.
3Action: ```json JSON
4[
5 {
6 "tool_name": "directly_answer",
7 "parameters": {}
8 }
9]
10```
11Answer: Hey, I'm doing well, thank you for asking! How can I help you today?
12Grounded answer: Hey, I'm doing well, thank you for asking! How can I help you today?
13
14> Finished chain.
15
16{'input': 'Hey how are you?',
17 'output': "Hey, I'm doing well, thank you for asking! How can I help you today?",
18 'intermediate_steps': []}