Cohere Embed on LangChain (Integration Guide)

Cohere supports various integrations with LangChain, a large language model (LLM) framework which allows you to quickly create applications based on Cohere’s models. This doc will guide you through how to leverage different Cohere embeddings with LangChain.

Prerequisites

Running Cohere embeddings with LangChain doesn’t require many prerequisites, consult the top-level document for more information.

Cohere Embeddings with LangChain

To use Cohere’s Embeddings with LangChain, create a CohereEmbedding object as follows (the available cohere embedding models are listed here):

PYTHON

1 from langchain_cohere import CohereEmbeddings
2 
3 # Define the Cohere embedding model
4 embeddings = CohereEmbeddings(
5     cohere_api_key="COHERE_API_KEY", model="embed-english-v3.0"
6 )
7 
8 # Embed a document
9 text = "This is a test document."
10 query_result = embeddings.embed_query(text)
11 print(query_result[:5], "...")
12 doc_result = embeddings.embed_documents([text])
13 print(doc_result[0][:5], "...")

To use these embeddings with Cohere’s RAG functionality, you will need to use one of the vector DBs from this list. In this example we use chroma, so in order to run it you will need to install chroma using pip install chromadb.

PYTHON

1 from langchain_cohere import (
2     ChatCohere,
3     CohereEmbeddings,
4     CohereRerank,
5     CohereRagRetriever,
6 )
7 from langchain.text_splitter import CharacterTextSplitter
8 from langchain_community.vectorstores import Chroma
9 from langchain_community.document_loaders import WebBaseLoader
10 
11 user_query = "what is Cohere Toolkit?"
12 
13 llm = ChatCohere(
14     cohere_api_key="COHERE_API_KEY",
15     model="command-r-plus-08-2024",
16     temperature=0,
17 )
18 
19 embeddings = CohereEmbeddings(
20     cohere_api_key="COHERE_API_KEY", model="embed-english-v3.0"
21 )
22 
23 # Load text files and split into chunks, you can also use data gathered elsewhere in your application
24 raw_documents = WebBaseLoader(
25     "https://docs.cohere.com/docs/cohere-toolkit"
26 ).load()
27 
28 text_splitter = CharacterTextSplitter(chunk_size=500, chunk_overlap=0)
29 documents = text_splitter.split_documents(raw_documents)
30 # Create a vector store from the documents
31 db = Chroma.from_documents(documents, embeddings)
32 input_docs = db.as_retriever().invoke(user_query)
33 
34 # Create the cohere rag retriever using the chat model
35 rag = CohereRagRetriever(llm=llm)
36 docs = rag.invoke(
37     user_query,
38     documents=input_docs,
39 )
40 # Print the documents
41 print("Documents:")
42 for doc in docs[:-1]:
43     print(doc.metadata)
44     print("\n\n" + doc.page_content)
45     print("\n\n" + "-" * 30 + "\n\n")
46 # Print the final generation
47 answer = docs[-1].page_content
48 print("Answer:")
49 print(answer)
50 # Print the final citations
51 citations = docs[-1].metadata["citations"]
52 print("Citations:")
53 print(citations)

Cohere with LangChain and Bedrock

Prerequisite

In addition to the prerequisites above, integrating Cohere with LangChain on Amazon Bedrock also requires:

The LangChain AWS package. To install it, run pip install langchain-aws.
AWS Python SDK. To install it, run pip install boto3. You can find more details here .
Configured authentication credentials for AWS. For more details, see this document.

Cohere Embeddings with LangChain and Amazon Bedrock

In this example, we create embeddings for a query using Bedrock and LangChain:

PYTHON

1 from langchain_aws import BedrockEmbeddings
2 
3 # Replace the profile name with the one created in the setup.
4 embeddings = BedrockEmbeddings(
5     credentials_profile_name="{PROFILE-NAME}",
6     region_name="us-east-1",
7     model_id="cohere.embed-english-v3",
8 )
9 
10 embeddings.embed_query("This is a content of the document")

Using LangChain on Private Deployments

You can use LangChain with privately deployed Cohere models. To use it, specify your model deployment URL in the base_url parameter.

PYTHON

1 llm = CohereEmbeddings(
2     base_url="<YOUR_DEPLOYMENT_URL>",
3     cohere_api_key="COHERE_API_KEY",
4     model="MODEL_NAME",
5 )

1	from langchain_cohere import CohereEmbeddings
2
3	# Define the Cohere embedding model
4	embeddings = CohereEmbeddings(
5	cohere_api_key="COHERE_API_KEY", model="embed-english-v3.0"
6	)
7
8	# Embed a document
9	text = "This is a test document."
10	query_result = embeddings.embed_query(text)
11	print(query_result[:5], "...")
12	doc_result = embeddings.embed_documents([text])
13	print(doc_result[0][:5], "...")

1	from langchain_cohere import (
2	ChatCohere,
3	CohereEmbeddings,
4	CohereRerank,
5	CohereRagRetriever,
6	)
7	from langchain.text_splitter import CharacterTextSplitter
8	from langchain_community.vectorstores import Chroma
9	from langchain_community.document_loaders import WebBaseLoader
10
11	user_query = "what is Cohere Toolkit?"
12
13	llm = ChatCohere(
14	cohere_api_key="COHERE_API_KEY",
15	model="command-r-plus-08-2024",
16	temperature=0,
17	)
18
19	embeddings = CohereEmbeddings(
20	cohere_api_key="COHERE_API_KEY", model="embed-english-v3.0"
21	)
22
23	# Load text files and split into chunks, you can also use data gathered elsewhere in your application
24	raw_documents = WebBaseLoader(
25	"https://docs.cohere.com/docs/cohere-toolkit"
26	).load()
27
28	text_splitter = CharacterTextSplitter(chunk_size=500, chunk_overlap=0)
29	documents = text_splitter.split_documents(raw_documents)
30	# Create a vector store from the documents
31	db = Chroma.from_documents(documents, embeddings)
32	input_docs = db.as_retriever().invoke(user_query)
33
34	# Create the cohere rag retriever using the chat model
35	rag = CohereRagRetriever(llm=llm)
36	docs = rag.invoke(
37	user_query,
38	documents=input_docs,
39	)
40	# Print the documents
41	print("Documents:")
42	for doc in docs[:-1]:
43	print(doc.metadata)
44	print("\n\n" + doc.page_content)
45	print("\n\n" + "-" * 30 + "\n\n")
46	# Print the final generation
47	answer = docs[-1].page_content
48	print("Answer:")
49	print(answer)
50	# Print the final citations
51	citations = docs[-1].metadata["citations"]
52	print("Citations:")
53	print(citations)

1	from langchain_aws import BedrockEmbeddings
2
3	# Replace the profile name with the one created in the setup.
4	embeddings = BedrockEmbeddings(
5	credentials_profile_name="{PROFILE-NAME}",
6	region_name="us-east-1",
7	model_id="cohere.embed-english-v3",
8	)
9
10	embeddings.embed_query("This is a content of the document")

1	llm = CohereEmbeddings(
2	base_url="<YOUR_DEPLOYMENT_URL>",
3	cohere_api_key="COHERE_API_KEY",
4	model="MODEL_NAME",
5	)