Calling an Encrypted Vault over the API

An Encrypted Vault uses the same endpoints and the same request and response shapes as a Standard Vault: the same chat, embed, and rerank calls. The one critical difference is that you cannot call the vault endpoint directly. Traffic must go through the Cohere OHTTP proxy, a client-side confidential-computing component that runs in your environment and, before any data leaves it, verifies the deployment’s remote attestation and encrypts your request end to end, refusing the connection if verification does not pass.

Two ways to connect through the Cohere OHTTP proxy

The Cohere OHTTP proxy comes in two forms, and which one you use determines which clients you can use:

In all cases, the Cohere OHTTP proxy does the same two things on your behalf:

• Verifies attestation: it fetches the environment’s attestation token and OHTTP public key, confirms the token was genuinely signed by Intel Trust Authority, checks that the attested measurements match the approved policy, and confirms the OHTTP key is bound to that attested environment.
• Encrypts end to end: it encrypts every request to the attested environment using OHTTP and decrypts the response, so the load balancer and network only ever see ciphertext.

Option 1: Cohere OHTTP proxy with any client

Run the Cohere OHTTP proxy container locally, then point any client at the local proxy address instead of the vault endpoint. The proxy verifies attestation and encrypts all traffic to the attested environment, so from your client’s perspective the call looks like an ordinary plaintext request to a local address. This means any clients producing HTTP Cohere/OpenAI API requests work without changes.

Run the container using a container platform such as Docker:

$
docker run -it --rm \
>
  -p 127.0.0.1:8080:8080 \
>
  -e IN_HOST=0.0.0.0 -e IN_PORT=8080 \
>
  -e TARGET_URL=<YOUR_VAULT_ENDPOINT_URL> \
>
  ghcr.io/cohere-ai/tng-ingress:latest

The latest tag always points at the current release. In production, pin a specific version (for example ghcr.io/cohere-ai/tng-ingress:0.6.0) so proxy updates only land when you choose them.

Then use the Cohere SDK via the proxy:

1
import cohere
2
3
# Point the SDK at the local Cohere OHTTP proxy instead of the vault endpoint.
4
co = cohere.ClientV2(
5
    api_key="<COHERE_API_KEY>",
6
    base_url="<LOCAL_OHTTP_PROXY_URL>",
7
)
8
9
response = co.chat(
10
    model="<YOUR_VAULT_MODEL_NAME>",
11
    messages=[
12
        {"role": "user", "content": "Hello to an encrypted vault!"}
13
    ],
14
)
15
16
print(response.message.content[0].text)

Make a raw HTTP request via the proxy:

$
curl "<LOCAL_OHTTP_PROXY_URL>/v2/chat" \
>
  -H "Authorization: Bearer <COHERE_API_KEY>" \
>
  -H "Content-Type: application/json" \
>
  -d '{
>
    "model": "<YOUR_VAULT_MODEL_NAME>",
>
    "messages": [{"role": "user", "content": "Hello to an encrypted vault!"}]
>
  }'

Use OpenAI-compatible client via the proxy:

1
from openai import OpenAI
2
3
client = OpenAI(
4
    api_key="<COHERE_API_KEY>",
5
    base_url="<LOCAL_OHTTP_PROXY_URL>/v1",
6
)
7
8
response = client.chat.completions.create(
9
    model="<YOUR_VAULT_MODEL_NAME>",
10
    messages=[
11
        {"role": "user", "content": "Hello to an encrypted vault!"}
12
    ],
13
)
14
15
print(response.choices[0].message.content)

Option 2: Cohere Python OHTTP Wrapper

Use our ‘conseel’ package to replace the transport layer in any httpx-based Python SDK with one that verifies attestation and encrypts all traffic to the attested environment. Besides the change in client construction, the SDK usage remains unchanged from a standard vault.

Install package via PyPI (for x86/ARM machines running Linux/Mac):

pip install conseel

Use with Cohere SDK:

1
import cohere
2
import httpx
3
from conseel import Transport
4
5
co = cohere.ClientV2(
6
    api_key="<COHERE_API_KEY>",
7
    base_url="<YOUR_VAULT_ENDPOINT_URL>",
8
    httpx_client=httpx.Client(transport=Transport()),
9
)
10
11
response = co.chat(
12
    model="<YOUR_VAULT_MODEL_NAME>",
13
    messages=[
14
        {"role": "user", "content": "Hello to an encrypted vault!"}
15
    ],
16
)
17
18
print(response.message.content[0].text)

Use with other httpx-based clients such as the OpenAI SDK:

1
import openai
2
import httpx
3
from conseel import Transport
4
5
client = openai.OpenAI(
6
    api_key="<COHERE_API_KEY>",
7
    base_url="<YOUR_VAULT_ENDPOINT_URL>/v1",
8
    httpx_client=httpx.Client(transport=Transport()),
9
)
10
11
response = client.chat.completions.create(
12
    model="<YOUR_VAULT_MODEL_NAME>",
13
    messages=[
14
        {"role": "user", "content": "Hello to an encrypted vault!"}
15
    ],
16
)
17
18
print(response.choices[0].message.content)

Verifying before send

The Cohere OHTTP proxy enforces a strict order: no plaintext leaves your environment until attestation passes. On each session it confirms the environment is genuine confidential-VM CPU and NVIDIA GPU hardware, that debug modes are off, that firmware and code measurements match the approved policy, and that the encryption key belongs to that environment. Only then does it encrypt and send your request. Attestation tokens are short-lived and refreshed periodically, so the guarantee reflects the environment’s current state.

Verification is not limited to setup time: With the Cohere Python OHTTP wrapper (conseel), every inference response includes an attestation certificate (under the x-tng-attestation-token header) that your client can check programmatically, so you can inspect proof of the policy, CPU, GPU, and software for the environment that served each response.

For the full list of what is checked and how to inspect it, see Verifying Your Deployment.

Handling attestation failures

If verification fails, the Cohere OHTTP proxy refuses the connection: it does not send your request, so your data is never exposed to an unverified environment. Common causes include an invalid token signature, a policy that did not match (the measured software differs from the approved stack), an expired token, or an encryption key that is not bound to the attested environment.

When this happens, confirm the vault’s status in the Model Vault app, and if it consistently fails verification, contact Cohere support. See Verifying Your Deployment for what each failure means.

Hosted Cohere OHTTP proxy for demos

Cohere also offers a hosted OHTTP proxy that allows you to interact with an encrypted vault as though it were a standard vault, without running any of the client-side options described above. This option is intended strictly for demo purposes, as it changes the security model by requiring you to trust Cohere with the proxy layer. To use the hosted OHTTP proxy, send requests to the encrypted vault as you would a standard vault and include the following header with all requests:

"X-Cohere-Demo-Encrypt": "1"

Related pages

• Calling a Standard Vault over the API
• Remote Attestation
• Verifying Your Deployment