Overview of RAG Connectors
As the name implies, Connectors are ways of connecting to data sources. They enable you to combine Cohere large language models (LLMs), which power the Chat API endpoint, with data sources such as internal documents, document databases, the broader internet, or any other source of context which can inform the replies generated by the model.
Connectors enhance Cohere retrieval augmented generation (RAG) offering and can respond to user questions and prompts with substantive, grounded generations that contain citations to external public or private knowledge bases. To see an example of grounded generations with citations, try out the Cohere dashboard after enabling web search grounding.
The following graphic demonstrates the flow of information when using a connector:
Using Connectors to Create Grounded Generations
Connectors are specified when calling the Chat endpoint, which you can read more about here. An example request specifying the managed web-search connector would look like this:
If you or an administrator at your organization has created a new connector, you can add this connector id to the list. Here’s an example:
The response will then contain the generated text with citation elements that link to the documents returned from the connector. For example, the formula C6H12O6
below has a citation element that links to three websites.
A Caveat on Deploying Connectors
Connector registration only works natively on the Cohere platform. You can, however, register a connector for e.g. Azure or another platform using the Cohere toolkit (more technical detail is available here.) You might also find it useful to read about Cohere deployments on Amazon, Azure, and single-container cloud environments.