Chat on LangChain

Cohere supports various integrations with LangChain, a large language model (LLM) framework which allows you to quickly create applications based on Cohere’s models. This doc will guide you through how to leverage Cohere Chat with LangChain.

Prerequisites

Running Cohere Chat with LangChain doesn’t require many prerequisites, consult the top-level document for more information.

Cohere Chat with LangChain

To use Cohere chat with LangChain, simply create a ChatCohere object and pass in the message or message history. In the example below, you will need to add your Cohere API key.

PYTHON
1from langchain_community.chat_models import ChatCohere
2from langchain_core.messages import AIMessage, HumanMessage
3
4cohere_chat_model = ChatCohere(cohere_api_key="{API_KEY}")
5
6# Send a chat message without chat history
7current_message = [HumanMessage(content="knock knock")]
8print(cohere_chat_model(current_message))
9
10# Send a chat message with chat history, note the last message is the current user message
11current_message_and_history = [
12 HumanMessage(content="knock knock"),
13 AIMessage(content="Who's there?"),
14 HumanMessage(content="Tank") ]
15print(cohere_chat_model(current_message_and_history))

Cohere Agents with LangChain

LangChain Agents use a language model to choose a sequence of actions to take.

To use Cohere’s multi hop agent create a create_cohere_react_agent and pass in the LangChain tools you would like to use.

For example, using an internet search tool to get essay writing advice from Cohere with citations:

PYTHON
1from langchain.agents import AgentExecutor
2from langchain_cohere.chat_models import ChatCohere
3from langchain_cohere.react_multi_hop.agent import create_cohere_react_agent
4from langchain_community.tools.tavily_search import TavilySearchResults
5from langchain_core.prompts import ChatPromptTemplate
6
7# Internet search tool - you can use any tool, and there are lots of community tools in LangChain.
8# To use the Tavily tool you will need to set an API key in the TAVILY_API_KEY environment variable.
9internet_search = TavilySearchResults()
10
11# Create and run the Cohere agent
12# Set a Cohere API key in the COHERE_API_KEY environment variable.
13llm = ChatCohere()
14agent = create_cohere_react_agent(
15 llm=llm,
16 tools=[internet_search],
17 prompt=ChatPromptTemplate.from_template("{question}"),
18)
19agent_executor = AgentExecutor(agent=agent, tools=[internet_search], verbose=True)
20
21response = agent_executor.invoke({
22 "question": "I want to write an essay. Any tips?",
23})
24# See Cohere's response
25print(response.get("output"))
26# Cohere provides exact citations for the sources it used
27print(response.get("citations"))

Cohere Chat and RAG with LangChain

To use Cohere’s retrieval augmented generation (RAG) functionality with LangChain, create a CohereRagRetriever object. Then there are a few RAG uses, discussed in the next few sections.

Using LangChain’s Retrievers

In this example, we use the wikipedia retriever but any retriever supported by LangChain can be used here. In order to set up the wikipedia retriever you need to install the wikipedia python package using %pip install --upgrade --quiet wikipedia. With that done, you can execute this code to see how a retriever works:

PYTHON
1from langchain.retrievers import CohereRagRetriever
2from langchain.retrievers import WikipediaRetriever
3from langchain_community.chat_models import ChatCohere
4
5# User query we will use for the generation
6user_query = "What is cohere?"
7# Load the cohere chat model
8cohere_chat_model = ChatCohere(cohere_api_key="{API_KEY}")
9# Create the cohere rag retriever using the chat model
10rag = CohereRagRetriever(llm=cohere_chat_model, connectors=[])
11# Create the wikipedia retriever
12wiki_retriever = WikipediaRetriever()
13# Get the relevant documents from wikipedia
14wiki_docs = wiki_retriever.get_relevant_documents(user_query )
15# Get the cohere generation from the cohere rag retriever
16docs = rag.get_relevant_documents(user_query ,source_documents=wiki_docs)
17# Print the documents
18for doc in docs[:-1]:
19 print(doc.metadata)
20 print("\n\n" + doc.page_content)
21 print("\n\n" + "-" * 30 + "\n\n")
22# Print the final generation
23answer = docs[-1].page_content
24print(answer)
25# Print the final citations
26citations = docs[-1].metadata['citations']
27print(citations)

Using Documents

In this example, we take documents (which might be generated in other parts of your application) and pass them into the CohereRagRetriever object:

PYTHON
1from langchain.retrievers import CohereRagRetriever
2from langchain_community.chat_models import ChatCohere
3from langchain_core.documents import Document
4
5# Load the cohere chat model
6cohere_chat_model = ChatCohere(cohere_api_key="{API_KEY}")
7# Create the cohere rag retriever using the chat model
8rag = CohereRagRetriever(llm=cohere_chat_model, connectors=[])
9docs = rag.get_relevant_documents(
10 "Does LangChain support cohere RAG?",
11 source_documents=[
12 Document(page_content="LangChain supports cohere RAG!", metadata={"id": "id-1"}),
13 Document(page_content="The sky is blue!", metadata={"id": "id-2"}),
14 ],
15)
16# Print the documents
17for doc in docs[:-1]:
18 print(doc.metadata)
19 print("\n\n" + doc.page_content)
20 print("\n\n" + "-" * 30 + "\n\n")
21# Print the final generation
22answer = docs[-1].page_content
23print(answer)
24# Print the final citations
25citations = docs[-1].metadata['citations']
26print(citations)

Using a Connector

In this example, we create a generation with a connector which allows us to get a generation with citations to results from the connector. We use the “web-search” connector, which is available to everyone. But if you have created your own connector in your org you can pass in its id, like so: rag = CohereRagRetriever(llm=cohere_chat_model, connectors=[{"id": "example-connector-id"}])

Here’s a code sample illustrating how to use a connector:

PYTHON
1from langchain.retrievers import CohereRagRetriever
2from langchain_community.chat_models import ChatCohere
3from langchain_core.documents import Document
4
5# Load the cohere chat model
6cohere_chat_model = ChatCohere(cohere_api_key="{API_KEY}")
7# Create the cohere rag retriever using the chat model with the web search connector
8rag = CohereRagRetriever(llm=cohere_chat_model, connectors=[{"id": "web-search"}])
9docs = rag.get_relevant_documents("Who founded Cohere?")
10# Print the documents
11for doc in docs[:-1]:
12 print(doc.metadata)
13 print("\n\n" + doc.page_content)
14 print("\n\n" + "-" * 30 + "\n\n")
15# Print the final generation
16answer = docs[-1].page_content
17print(answer)
18# Print the final citations
19citations = docs[-1].metadata['citations']
20print(citations)