Cohere on Azure

Note

The code examples in this section use the Cohere v1 API. The v2 API is not yet supported for cloud deployments and will be coming soon.

In an effort to make our language-model capabilities more widely available, we’ve partnered with a few major platforms to create hosted versions of our offerings.

In this article, you learn how to use Azure AI Studio to deploy both the Cohere Command models and the Cohere Embed models on Microsoft’s Azure cloud computing platform.

The following six models are available through Azure AI Studio with pay-as-you-go, token-based billing:

Prerequisites

Whether you’re using Command or Embed, the initial set up is the same. You’ll need:

  • An Azure subscription with a valid payment method. Free or trial Azure subscriptions won’t work. If you don’t have an Azure subscription, create a paid Azure account to begin.
  • An Azure AI hub resource. Note: for Cohere models, the pay-as-you-go deployment offering is only available with AI hubs created in the EastUS, EastUS2 or Sweden Central regions.
  • An Azure AI project in Azure AI Studio.
  • Azure role-based access controls (Azure RBAC) are used to grant access to operations in Azure AI Studio. To perform the required steps, your user account must be assigned the Azure AI Developer role on the resource group. For more information on permissions, see Role-based access control in Azure AI Studio.

For workflows based around Command, Embed, or Rerank, you’ll also need to create a deployment and consume the model. Here are links for more information:

Text Generation

We expose two routes for Command R and Command R+ inference:

  • v1/chat/completions adheres to the Azure AI Generative Messages API schema;
  • v1/chat supports Cohere’s native API schema.

You can find more information about Azure’s API here.

Here’s a code snippet demonstrating how to programmatically interact with a Cohere model on Azure:

PYTHON
1import urllib.request
2import json
3
4# Configure payload data sending to API endpoint
5data = {
6    "messages": [
7        {"role": "system", "content": "You are a helpful assistant."},
8        {"role": "user", "content": "What is good about Wuhan?"},
9    ],
10    "max_tokens": 500,
11    "temperature": 0.3,
12    "stream": "True",
13}
14
15body = str.encode(json.dumps(data))
16
17# Replace the url with your API endpoint
18url = "https://your-endpoint.inference.ai.azure.com/v1/chat/completions"
19
20# Replace this with the key for the endpoint
21api_key = "your-auth-key"
22if not api_key:
23    raise Exception("API Key is missing")
24
25headers = {"Content-Type": "application/json", "Authorization": (api_key)}
26
27req = urllib.request.Request(url, body, headers)
28
29try:
30    response = urllib.request.urlopen(req)
31    result = response.read()
32    print(result)
33except urllib.error.HTTPError as error:
34    print("The request failed with status code: " + str(error.code))
35    # Print the headers - they include the requert ID and the timestamp, which are useful for debugging the failure
36    print(error.info())
37    print(error.read().decode("utf8", "ignore"))

You can find more code snippets, including examples of how to stream responses, in this notebook.

Though this section is called “Text Generation”, it’s worth pointing out that these models are capable of much more. Specifically, you can use Azure-hosted Cohere models for both retrieval augmented generation and multi-step tool use. Check the linked pages for much more information.

Finally, we released refreshed versions of Command R and Command R+ in August 2024, both of which are now available on Azure. Check these Microsoft docs for more information (select the Cohere Command R 08-2024 or Cohere Command R+ 08-2024 tabs).

Embeddings

We expose two routes for Embed v3 - English and Embed v3 - Multilingual inference:

  • v1/embeddings adheres to the Azure AI Generative Messages API schema;
  • v1/embed supports Cohere’s native API schema.

You can find more information about Azure’s API here.

PYTHON
1import urllib.request
2import json
3
4# Configure payload data sending to API endpoint
5data = {
6    "input": ["hi"]
7}
8
9body = str.encode(json.dumps(data))
10
11# Replace the url with your API endpoint
12url = "https://your-endpoint.inference.ai.azure.com/v1/embedding"
13
14# Replace this with the key for the endpoint
15api_key = "your-auth-key"
16if not api_key:
17    raise Exception("API Key is missing")
18
19headers = {"Content-Type": "application/json", "Authorization": (api_key)}
20
21req = urllib.request.Request(url, body, headers)
22
23try:
24    response = urllib.request.urlopen(req)
25    result = response.read()
26    print(result)
27except urllib.error.HTTPError as error:
28    print("The request failed with status code: " + str(error.code))
29    # Print the headers - they include the requert ID and the timestamp, which are useful for debugging the failure
30    print(error.info())
31    print(error.read().decode("utf8", "ignore"))

ReRank

We currently exposes the v1/rerank endpoint for inference with both Rerank 3 - English and Rerank 3 - Multilingual. For more information on using the APIs, see the reference section.

PYTHON
1import cohere
2
3co = cohere.Client(
4    base_url="https://<endpoint>.<region>.inference.ai.azure.com/v1",
5    api_key="<key>"
6)
7
8documents = [
9    {
10        "Title": "Incorrect Password",
11        "Content": "Hello, I have been trying to access my account for the past hour and it keeps saying my password is incorrect. Can you please help me?",
12    },
13    {
14        "Title": "Confirmation Email Missed",
15        "Content": "Hi, I recently purchased a product from your website but I never received a confirmation email. Can you please look into this for me?",
16    },
17    {
18        "Title": "Questions about Return Policy",
19        "Content": "Hello, I have a question about the return policy for this product. I purchased it a few weeks ago and it is defective.",
20    },
21    {
22        "Title": "Customer Support is Busy",
23        "Content": "Good morning, I have been trying to reach your customer support team for the past week but I keep getting a busy signal. Can you please help me?",
24    },
25    {
26        "Title": "Received Wrong Item",
27        "Content": "Hi, I have a question about my recent order. I received the wrong item and I need to return it.",
28    },
29    {
30        "Title": "Customer Service is Unavailable",
31        "Content": "Hello, I have been trying to reach your customer support team for the past hour but I keep getting a busy signal. Can you please help me?",
32    },
33    {
34        "Title": "Return Policy for Defective Product",
35        "Content": "Hi, I have a question about the return policy for this product. I purchased it a few weeks ago and it is defective.",
36    },
37    {
38        "Title": "Wrong Item Received",
39        "Content": "Good morning, I have a question about my recent order. I received the wrong item and I need to return it.",
40    },
41    {
42        "Title": "Return Defective Product",
43        "Content": "Hello, I have a question about the return policy for this product. I purchased it a few weeks ago and it is defective.",
44    },
45]
46
47response = co.rerank(
48    documents=documents,
49    query="What emails have been about returning items?",
50    rank_fields=["Title", "Content"],
51    top_n=5,
52)

A Note on SDKs

You should be aware that it’s possible to use the cohere SDK client to consume Azure AI deployments. Here are example notes for Command and Embed.

The important thing to understand is that our new and existing customers can call the models from Azure while still leveraging their integration with the Cohere SDK.