Semantic Search

Cohere’s embedding models are available via the Embed endpoint. This endpoint enables you to embed text documents (multilingual) and images into a vector space.

Semantic search, powered by embeddings, enables applications to perform information retrieval based on the context or meaning of a document.

This quickstart guide shows you how to perform semantic search with the Embed endpoint.

1

Setup

First, install the Cohere Python SDK with the following command.

$pip install -U cohere

Next, import the library and create a client.

PYTHON
1import cohere
2
3co = cohere.Client(
4    "COHERE_API_KEY"
5)  # Get your free API key here: https://dashboard.cohere.com/api-keys
2

Document Embeddings

First, embed the list of available documents using the Embed endpoint by specifying the input_type as search_document.

PYTHON
1# Define the documents
2documents = [
3    "Joining Slack Channels: Be sure to join relevant channels to stay informed and engaged.",
4    "Finding Coffee Spots: For your caffeine fix, cross the street to the café for artisan coffee.",
5    "Working Hours Flexibility: While our core hours are 9 AM to 5 PM, we offer flexibility to adjust as needed.",
6]
7
8# Embed the documents
9doc_emb = co.embed(
10    model="embed-v4.0",
11    input_type="search_document",
12    texts=documents,
13    max_tokens=8000,
14    output_dimension=1024,
15    embedding_types=["float"],
16).embeddings.float
3

Query Embedding

Next, embed the user query using the Embed endpoint by specifying the input_type as search_query.

PYTHON
1# Add the user query
2query = "Ways to connect with my teammates"
3
4# Embed the query
5query_emb = co.embed(
6    model="embed-v4.0",
7    input_type="search_query",
8    texts=[query],
9    max_tokens=8000,
10    output_dimension=1024,
11    embedding_types=["float"],
12).embeddings.float
4

Semantic Search

Then, perform semantic search by computing the similarity between the query embedding and the document embeddings, and then returning the most similar documents.

PYTHON
1import numpy as np
2
3
4# Compute dot product similarity and display results
5def return_results(query_emb, doc_emb, documents):
6    n = 2  # customize your top N results
7    scores = np.dot(query_emb, np.transpose(doc_emb))[0]
8    max_idx = np.argsort(-scores)[:n]
9
10    for rank, idx in enumerate(max_idx):
11        print(f"Rank: {rank+1}")
12        print(f"Score: {scores[idx]}")
13        print(f"Document: {documents[idx]}\n")
14
15
16return_results(query_emb, doc_emb, documents)
1Rank: 1
2Score: 0.262197161387274
3Document: Joining Slack Channels: Be sure to join relevant channels to stay informed and engaged.
4
5Rank: 2
6Score: 0.1266074257723145
7Document: Working Hours Flexibility: While our core hours are 9 AM to 5 PM, we offer flexibility to adjust as needed.

Further Resources

Built with