Chat on LangChain

Cohere supports various integrations with LangChain, a large language model (LLM) framework which allows you to quickly create applications based on Cohere's models. This doc will guide you through how to leverage Cohere Chat with LangChain.

Prerequisites

Running Cohere Chat with LangChain doesn't require many prerequisites, consult the top-level document for more information.

Cohere Chat with LangChain

To use Cohere chat with LangChain, simply create a ChatCohere object and pass in the message or message history. In the example below, you will need to add your Cohere API key.

from langchain_community.chat_models import ChatCohere
from langchain_core.messages import AIMessage, HumanMessage

cohere_chat_model = ChatCohere(cohere_api_key="{API_KEY}")

# Send a chat message without chat history
current_message = [HumanMessage(content="knock knock")]
print(cohere_chat_model(current_message))

# Send a chat message with chat history, note the last message is the current user message
current_message_and_history = [
    HumanMessage(content="knock knock"),
    AIMessage(content="Who's there?"),
    HumanMessage(content="Tank") ]
print(cohere_chat_model(current_message_and_history))

Cohere Agents with LangChain

LangChain Agents use a language model to choose a sequence of actions to take.

To use Cohere's multi hop agent create a create_cohere_react_agent and pass in the LangChain tools you would like to use.

For example, using an internet search tool to get essay writing advice from Cohere with citations:

from langchain.agents import AgentExecutor
from langchain_cohere.chat_models import ChatCohere
from langchain_cohere.react_multi_hop.agent import create_cohere_react_agent
from langchain_community.tools.tavily_search import TavilySearchResults
from langchain_core.prompts import ChatPromptTemplate

# Internet search tool - you can use any tool, and there are lots of community tools in LangChain.
# To use the Tavily tool you will need to set an API key in the TAVILY_API_KEY environment variable.
internet_search = TavilySearchResults()

# Create and run the Cohere agent
# Set a Cohere API key in the COHERE_API_KEY environment variable.
llm = ChatCohere()
agent = create_cohere_react_agent(
    llm=llm,
    tools=[internet_search],
    prompt=ChatPromptTemplate.from_template("{question}"),
)
agent_executor = AgentExecutor(agent=agent, tools=[internet_search], verbose=True)

response = agent_executor.invoke({
    "question": "I want to write an essay. Any tips?",
})
# See Cohere's response
print(response.get("output"))
# Cohere provides exact citations for the sources it used
print(response.get("citations"))

Cohere Chat and RAG with LangChain

To use Cohere's retrieval augmented generation (RAG) functionality with LangChain, create a CohereRagRetriever object. Then there are a few RAG uses, discussed in the next few sections.

Using LangChain's Retrievers

In this example, we use the wikipedia retriever but any retriever supported by LangChain can be used here. In order to set up the wikipedia retriever you need to install the wikipedia python package using %pip install --upgrade --quiet wikipedia. With that done, you can execute this code to see how a retriever works:

from langchain.retrievers import CohereRagRetriever
from langchain.retrievers import WikipediaRetriever
from langchain_community.chat_models import ChatCohere

# User query we will use for the generation
user_query = "What is cohere?"
# Load the cohere chat model
cohere_chat_model = ChatCohere(cohere_api_key="{API_KEY}")
# Create the cohere rag retriever using the chat model
rag = CohereRagRetriever(llm=cohere_chat_model, connectors=[])
# Create the wikipedia retriever
wiki_retriever = WikipediaRetriever()
# Get the relevant documents from wikipedia
wiki_docs = wiki_retriever.get_relevant_documents(user_query )
# Get the cohere generation from the cohere rag retriever
docs = rag.get_relevant_documents(user_query ,source_documents=wiki_docs)
# Print the documents
for doc in docs[:-1]:
    print(doc.metadata)
    print("\n\n" + doc.page_content)
    print("\n\n" + "-" * 30 + "\n\n")
# Print the final generation 
answer = docs[-1].page_content
print(answer)
# Print the final citations 
citations = docs[-1].metadata['citations']
print(citations)

Using Documents

In this example, we take documents (which might be generated in other parts of your application) and pass them into the CohereRagRetriever object:

from langchain.retrievers import CohereRagRetriever
from langchain_community.chat_models import ChatCohere
from langchain_core.documents import Document

# Load the cohere chat model
cohere_chat_model = ChatCohere(cohere_api_key="{API_KEY}")
# Create the cohere rag retriever using the chat model
rag = CohereRagRetriever(llm=cohere_chat_model, connectors=[])
docs = rag.get_relevant_documents(
    "Does LangChain support cohere RAG?",
    source_documents=[
        Document(page_content="LangChain supports cohere RAG!", metadata={"id": "id-1"}),
        Document(page_content="The sky is blue!", metadata={"id": "id-2"}),
    ],
)
# Print the documents
for doc in docs[:-1]:
    print(doc.metadata)
    print("\n\n" + doc.page_content)
    print("\n\n" + "-" * 30 + "\n\n")
# Print the final generation 
answer = docs[-1].page_content
print(answer)
# Print the final citations 
citations = docs[-1].metadata['citations']
print(citations)

Using a Connector

In this example, we create a generation with a connector which allows us to get a generation with citations to results from the connector. We use the "web-search" connector, which is available to everyone. But if you have created your own connector in your org you can pass in its id, like so: rag = CohereRagRetriever(llm=cohere_chat_model, connectors=[{"id": "example-connector-id"}])

Here's a code sample illustrating how to use a connector:

from langchain.retrievers import CohereRagRetriever
from langchain_community.chat_models import ChatCohere
from langchain_core.documents import Document

# Load the cohere chat model
cohere_chat_model = ChatCohere(cohere_api_key="{API_KEY}")
# Create the cohere rag retriever using the chat model with the web search connector
rag = CohereRagRetriever(llm=cohere_chat_model, connectors=[{"id": "web-search"}])
docs = rag.get_relevant_documents("Who founded Cohere?")
# Print the documents
for doc in docs[:-1]:
    print(doc.metadata)
    print("\n\n" + doc.page_content)
    print("\n\n" + "-" * 30 + "\n\n")
# Print the final generation 
answer = docs[-1].page_content
print(answer)
# Print the final citations 
citations = docs[-1].metadata['citations']
print(citations)