IntegrationsLangChain

Chat on LangChain

Cohere supports various integrations with LangChain, a large language model (LLM) framework which allows you to quickly create applications based on Cohere’s models. This doc will guide you through how to leverage Cohere Chat with LangChain.

Prerequisites

Running Cohere Chat with LangChain doesn’t require many prerequisites, consult the top-level document for more information.

Cohere Chat with LangChain

To use Cohere chat with LangChain, simply create a ChatCohere object and pass in the message or message history. In the example below, you will need to add your Cohere API key.

PYTHON
1from langchain_cohere import ChatCohere
2from langchain_core.messages import AIMessage, HumanMessage
3
4# Define the Cohere LLM
5llm = ChatCohere(cohere_api_key="COHERE_API_KEY",
6 model="command-r-plus-08-2024")
7
8# Send a chat message without chat history
9current_message = [HumanMessage(content="knock knock")]
10print(llm.invoke(current_message))
11
12# Send a chat message with chat history, note the last message is the current user message
13current_message_and_history = [
14 HumanMessage(content="knock knock"),
15 AIMessage(content="Who's there?"),
16 HumanMessage(content="Tank") ]
17print(llm.invoke(current_message_and_history))

Cohere Agents with LangChain

LangChain Agents use a language model to choose a sequence of actions to take.

To use Cohere’s multi hop agent create a create_cohere_react_agent and pass in the LangChain tools you would like to use.

For example, using an internet search tool to get essay writing advice from Cohere with citations:

PYTHON
1from langchain_cohere import ChatCohere
2from langchain_cohere.react_multi_hop.agent import create_cohere_react_agent
3from langchain.agents import AgentExecutor
4from langchain_community.tools.tavily_search import TavilySearchResults
5from langchain_core.prompts import ChatPromptTemplate
6
7# Internet search tool - you can use any tool, and there are lots of community tools in LangChain.
8# To use the Tavily tool you will need to set an API key in the TAVILY_API_KEY environment variable.
9os.environ["TAVILY_API_KEY"] = "TAVILY_API_KEY"
10internet_search = TavilySearchResults()
11
12# Define the Cohere LLM
13llm = ChatCohere(cohere_api_key="COHERE_API_KEY",
14 model="command-r-plus-08-2024")
15
16# Create an agent
17agent = create_cohere_react_agent(
18 llm=llm,
19 tools=[internet_search],
20 prompt=ChatPromptTemplate.from_template("{question}"),
21)
22
23# Create an agent executor
24agent_executor = AgentExecutor(agent=agent, tools=[internet_search], verbose=True)
25
26# Generate a response
27response = agent_executor.invoke(
28 {
29 "question": "I want to write an essay. Any tips?",
30 }
31)
32
33# See Cohere's response
34print(response.get("output"))
35# Cohere provides exact citations for the sources it used
36print(response.get("citations"))

Cohere Chat and RAG with LangChain

To use Cohere’s retrieval augmented generation (RAG) functionality with LangChain, create a CohereRagRetriever object. Then there are a few RAG uses, discussed in the next few sections.

Using LangChain’s Retrievers

In this example, we use the wikipedia retriever but any retriever supported by LangChain can be used here. In order to set up the wikipedia retriever you need to install the wikipedia python package using %pip install --upgrade --quiet wikipedia. With that done, you can execute this code to see how a retriever works:

PYTHON
1from langchain_cohere import CohereRagRetriever
2from langchain.retrievers import WikipediaRetriever
3from langchain_cohere import ChatCohere
4
5# User query we will use for the generation
6user_query = "What is cohere?"
7# Define the Cohere LLM
8llm = ChatCohere(cohere_api_key="COHERE_API_KEY",
9 model="command-r-plus-08-2024")
10# Create the Cohere rag retriever using the chat model
11rag = CohereRagRetriever(llm=llm, connectors=[])
12# Create the wikipedia retriever
13wiki_retriever = WikipediaRetriever()
14# Get the relevant documents from wikipedia
15wiki_docs = wiki_retriever.invoke(user_query)
16# Get the cohere generation from the cohere rag retriever
17docs = rag.invoke(user_query, documents=wiki_docs)
18# Print the documents
19print("Documents:")
20for doc in docs[:-1]:
21 print(doc.metadata)
22 print("\n\n" + doc.page_content)
23 print("\n\n" + "-" * 30 + "\n\n")
24# Print the final generation
25answer = docs[-1].page_content
26print("Answer:")
27print(answer)
28# Print the final citations
29citations = docs[-1].metadata["citations"]
30print("Citations:")
31print(docs[-1].__dict__)

Using Documents

In this example, we take documents (which might be generated in other parts of your application) and pass them into the CohereRagRetriever object:

PYTHON
1from langchain_cohere import CohereRagRetriever
2from langchain_cohere import ChatCohere
3from langchain_core.documents import Document
4
5# Define the Cohere LLM
6llm = ChatCohere(cohere_api_key="COHERE_API_KEY",
7 model="command-r-plus-08-2024")
8
9# Create the Cohere rag retriever using the chat model
10rag = CohereRagRetriever(llm=llm, connectors=[])
11docs = rag.invoke(
12 "Does LangChain support cohere RAG?",
13 documents=[
14 Document(page_content="LangChain supports cohere RAG!", metadata={"id": "id-1"}),
15 Document(page_content="The sky is blue!", metadata={"id": "id-2"}),
16 ],
17)
18
19# Print the documents
20print("Documents:")
21for doc in docs[:-1]:
22 print(doc.metadata)
23 print("\n\n" + doc.page_content)
24 print("\n\n" + "-" * 30 + "\n\n")
25# Print the final generation
26answer = docs[-1].page_content
27print("Answer:")
28print(answer)
29# Print the final citations
30citations = docs[-1].metadata['citations']
31print("Citations:")
32print(citations)

Using a Connector

In this example, we create a generation with a connector which allows us to get a generation with citations to results from the connector. We use the “web-search” connector, which is available to everyone. But if you have created your own connector in your org you can pass in its id, like so: rag = CohereRagRetriever(llm=cohere_chat_model, connectors=[{"id": "example-connector-id"}])

Here’s a code sample illustrating how to use a connector:

PYTHON
1from langchain_cohere import CohereRagRetriever
2from langchain_cohere import ChatCohere
3from langchain_core.documents import Document
4
5# Define the Cohere LLM
6llm = ChatCohere(cohere_api_key="COHERE_API_KEY",
7 model="command-r-plus-08-2024")
8
9# Create the Cohere rag retriever using the chat model with the web search connector
10rag = CohereRagRetriever(llm=llm, connectors=[{"id": "web-search"}])
11docs = rag.invoke("Who founded Cohere?")
12# Print the documents
13print("Documents:")
14for doc in docs[:-1]:
15 print(doc.metadata)
16 print("\n\n" + doc.page_content)
17 print("\n\n" + "-" * 30 + "\n\n")
18# Print the final generation
19answer = docs[-1].page_content
20print("Answer:")
21print(answer)
22# Print the final citations
23citations = docs[-1].metadata['citations']
24print("Citations:")
25print(citations)

Using the create_stuff_documents_chain Chain

This chain takes a list of documents and formats them all into a prompt, then passes that prompt to an LLM. It passes ALL documents, so you should make sure it fits within the context window of the LLM you are using.

Note: this feature is currently in beta.

PYTHON
1from langchain_cohere import ChatCohere
2from langchain_core.documents import Document
3from langchain_core.prompts import ChatPromptTemplate
4from langchain.chains.combine_documents import create_stuff_documents_chain
5
6prompt = ChatPromptTemplate.from_messages(
7 [("human", "What are everyone's favorite colors:\n\n{context}")]
8)
9
10# Define the Cohere LLM
11llm = ChatCohere(cohere_api_key="COHERE_API_KEY",
12 model="command-r-plus-08-2024")
13
14chain = create_stuff_documents_chain(llm, prompt)
15
16docs = [
17 Document(page_content="Jesse loves red but not yellow"),
18 Document(page_content = "Jamal loves green but not as much as he loves orange")
19]
20
21chain.invoke({"context": docs})

Structured Output Generation

Cohere supports generating JSON objects to structure and organize the model’s responses in a way that can be used in downstream applications.

You can specify the response_format parameter to indicate that you want the response in a JSON object format.

PYTHON
1from langchain_cohere import ChatCohere
2
3# Define the Cohere LLM
4llm = ChatCohere(cohere_api_key="COHERE_API_KEY",
5 model="command-r-plus-08-2024")
6
7res = llm.invoke("John is five years old", response_format={
8 "type": "json_object",
9 "schema": {
10 "title": "Person",
11 "description": "Identifies the age and name of a person",
12 "type": "object",
13 "properties": {
14 "name": { "type": "string", "description": "Name of the person" },
15 "age": { "type": "number", "description": "Age of the person" },
16 },
17 "required": [
18 "name",
19 "age",
20 ],
21 }
22 }
23)
24
25print(res)

Text Summarization

You can use the load_summarize_chain chain to perform text summarization.

PYTHON
1from langchain_cohere import ChatCohere
2from langchain.chains.summarize import load_summarize_chain
3from langchain_community.document_loaders import WebBaseLoader
4
5loader = WebBaseLoader("https://docs.cohere.com/docs/cohere-toolkit")
6docs = loader.load()
7
8# Define the Cohere LLM
9llm = ChatCohere(cohere_api_key="COHERE_API_KEY",
10 model="command-r-plus-08-2024",
11 temperature=0)
12
13chain = load_summarize_chain(llm, chain_type="stuff")
14
15chain.invoke({"input_documents": docs})

Using LangChain on Private Deployments

You can use LangChain with privately deployed Cohere models. To use it, specify your model deployment URL in the base_url parameter.

PYTHON
1llm = ChatCohere(base_url=<YOUR_DEPLOYMENT_URL>,
2 cohere_api_key="COHERE_API_KEY",
3 model="MODEL_NAME")