Multilingual Embedding Models

At Cohere, we are committed to breaking down barriers and expanding access to cutting-edge NLP technologies that power projects across the globe. By making our innovative multilingual language models available to all developers, we continue to move toward our goal of empowering developers, researchers, and innovators with state-of-the-art NLP technologies that push the boundaries of Language AI.

Our Multilingual Model maps text to a semantic vector space, positioning text with a similar meaning in close proximity. This process unlocks a range of valuable use cases for multilingual settings. For example, one can map a query to this vector space during a search to locate relevant documents nearby. This often yields search results that are several times better than keyword search.

Differences Between English and Multilingual Embedding Models

Unlike our English language embedding model, our multilingual model was trained using dot product calculations. Using dot products produces a non-normalized similarity score, reflecting the magnitude of the two compared vectors. When this dimension is incorporated, multilingual embeddings perform better than standard. For more information on how our English language model works (using cosine similarity), see our introductory guide to the Cohere platform. The dimensions of our multilingual embeddings is 768 dimensions.

Use Cases

• Multilingual Semantic Search: Improve your search results regardless of the language.
• Aggregate Customer Feedback: Organize customer feedback across hundreds of languages, simplifying a major challenge for international operations.
• Cross-Lingual Zero-Shot Content Moderation: Identify harmful content in online communities is challenging, especially as users speak hundreds of languages. Train a model with a few English examples, then detect harmful content in 100+ languages.

Get Started

To get started using the multilingual embedding models, you can either query our endpoints or install our SDK to use the model within Python:

import cohere  
co = cohere.Client(f"{api_key}")  
texts = [  
   'Hello from Cohere!', 'مرحبًا من كوهير!', 'Hallo von Cohere!',  
   'Bonjour de Cohere!', '¡Hola desde Cohere!', 'Olá do Cohere!',  
   'Ciao da Cohere!', '您好，来自 Cohere！', 'कोहेरे से नमस्ते!'  
]  
response = co.embed(texts=texts, model='embed-multilingual-v2.0')  
embeddings = response.embeddings # All text embeddings 
print(embeddings[0][:5]) # Print embeddings for the first text

Model Performance

List of Supported Languages

Our multilingual embedding model supports over 100 languages, including Chinese, Spanish, and French. For a full list of languages we support, please reference this page.