Rerank Overview
How Rerank Works
The Rerank API endpoint, powered by the Rerank models, is a simple and very powerful tool for semantic search. Given a query
and a list of documents
, Rerank indexes the documents from most to least semantically relevant to the query.
Get Started
Example with Texts
In the example below, we use the Rerank API endpoint to index the list of docs
from most to least relevant to the query What is the capital of the United States?
.
Request
In this example, the documents being passed in are a list of strings:
1 import cohere 2 co = cohere.ClientV2(api_key="<YOUR API KEY>") 3 4 query = "What is the capital of the United States?" 5 docs = [ 6 "Carson City is the capital city of the American state of Nevada. At the 2010 United States Census, Carson City had a population of 55,274.", 7 "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean that are a political division controlled by the United States. Its capital is Saipan.", 8 "Charlotte Amalie is the capital and largest city of the United States Virgin Islands. It has about 20,000 people. The city is on the island of Saint Thomas.", 9 "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district. The President of the USA and many major national government offices are in the territory. This makes it the political center of the United States of America.", 10 "Capital punishment (the death penalty) has existed in the United States since before the United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states. The federal government (including the United States military) also uses capital punishment."] 11 results = co.rerank(model="rerank-english-v3.0", query=query, documents=docs, top_n=5, return_documents=True)
Response
1 { 2 "id": "97813271-fe74-465d-b9d5-577e77079253", 3 "results": [ 4 { 5 "document": { 6 "text": "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district. The President of the USA and many major national government offices are in the territory. This makes it the political center of the United States of America." 7 }, 8 "index": 3, 9 "relevance_score": 0.9990564 10 }, 11 { 12 "document": { 13 "text": "Capital punishment (the death penalty) has existed in the United States since before the United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states. The federal government (including the United States military) also uses capital punishment." 14 }, 15 "index": 4, 16 "relevance_score": 0.7516481 17 }, 18 { 19 "document": { 20 "text": "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean that are a political division controlled by the United States. Its capital is Saipan." 21 }, 22 "index": 1, 23 "relevance_score": 0.08882029 24 }, 25 { 26 "document": { 27 "text": "Carson City is the capital city of the American state of Nevada. At the 2010 United States Census, Carson City had a population of 55,274." 28 }, 29 "index": 0, 30 "relevance_score": 0.058238626 31 }, 32 { 33 "document": { 34 "text": "Charlotte Amalie is the capital and largest city of the United States Virgin Islands. It has about 20,000 people. The city is on the island of Saint Thomas." 35 }, 36 "index": 2, 37 "relevance_score": 0.019946935 38 } 39 ], 40 "meta": { 41 "api_version": { 42 "version": "2022-12-06" 43 }, 44 "billed_units": { 45 "search_units": 1 46 } 47 } 48 }
Example with Semi-structured Data:
Alternatively, you can pass in a JSON object and specify the fields you’d like to rank over. If you do not pass in any rank_fields
, it will default to the text key.
Request
1 query = "What is the capital of the United States?" 2 docs = [ 3 {"Title":"Facts about Carson City","Content":"Carson City is the capital city of the American state of Nevada. At the 2010 United States Census, Carson City had a population of 55,274."}, 4 {"Title":"The Commonwealth of Northern Mariana Islands","Content":"The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean that are a political division controlled by the United States. Its capital is Saipan."}, 5 {"Title":"The Capital of United States Virgin Islands","Content":"Charlotte Amalie is the capital and largest city of the United States Virgin Islands. It has about 20,000 people. The city is on the island of Saint Thomas."}, 6 {"Title":"Washington D.C.","Content":"Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district. The President of the USA and many major national government offices are in the territory. This makes it the political center of the United States of America."}, 7 {"Title":"Capital Punishment in the US","Content":"Capital punishment (the death penalty) has existed in the United States since before the United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states. The federal government (including the United States military) also uses capital punishment."}] 8 results = co.rerank(model="rerank-english-v3.0", query=query, documents=docs, rank_fields=['Title','Content'],top_n=5, return_documents=True)
In the docs
parameter, we are passing in a list of objects which have the key values: [Title ,Content]
. As part of the Rerank call, we are specifying which keys to rank over, as well as the order in which the key value pairs should be considered.
1 { 2 "id": "75a94aa7-6761-4a64-a2ae-4bc0a62bc601", 3 "results": [ 4 { 5 "document": { 6 "Content": "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district. The President of the USA and many major national government offices are in the territory. This makes it the political center of the United States of America.", 7 "Title": "Washington D.C." 8 }, 9 "index": 3, 10 "relevance_score": 0.9987405 11 }, 12 { 13 "document": { 14 "Content": "Capital punishment (the death penalty) has existed in the United States since before the United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states. The federal government (including the United States military) also uses capital punishment.", 15 "Title": "Capital Punishment in the US" 16 }, 17 "index": 4, 18 "relevance_score": 0.5011778 19 }, 20 { 21 "document": { 22 "Content": "Charlotte Amalie is the capital and largest city of the United States Virgin Islands. It has about 20,000 people. The city is on the island of Saint Thomas.", 23 "Title": "The Capital of United States Virgin Islands" 24 }, 25 "index": 2, 26 "relevance_score": 0.10070161 27 }, 28 { 29 "document": { 30 "Content": "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean that are a political division controlled by the United States. Its capital is Saipan.", 31 "Title": "The Commonwealth of Northern Mariana Islands" 32 }, 33 "index": 1, 34 "relevance_score": 0.03197956 35 }, 36 { 37 "document": { 38 "Content": "Carson City is the capital city of the American state of Nevada. At the 2010 United States Census, Carson City had a population of 55,274.", 39 "Title": "Facts about Carson City" 40 }, 41 "index": 0, 42 "relevance_score": 0.019456575 43 } 44 ], 45 "meta": { 46 "api_version": { 47 "version": "2022-12-06" 48 }, 49 "billed_units": { 50 "search_units": 1 51 } 52 } 53 }
Multilingual Reranking
Cohere offers a multilingual model, rerank-multilingual-v3.0
. Please note that performance may vary across languages. The model is trained on the following languages:
ISO Code | Language Name |
---|---|
af | Afrikaans |
am | Amharic |
ar | Arabic |
as | Assamese |
az | Azerbaijani |
be | Belarusian |
bg | Bulgarian |
bn | Bengali |
bo | Tibetan |
bs | Bosnian |
ca | Catalan |
ceb | Cebuano |
co | Corsican |
cs | Czech |
cy | Welsh |
da | Danish |
de | German |
el | Greek |
en | English |
eo | Esperanto |
es | Spanish |
et | Estonian |
eu | Basque |
fa | Persian |
fi | Finnish |
fr | French |
fy | Frisian |
ga | Irish |
gd | Scots_gaelic |
gl | Galician |
gu | Gujarati |
ha | Hausa |
haw | Hawaiian |
he | Hebrew |
hi | Hindi |
hmn | Hmong |
hr | Croatian |
ht | Haitian_creole |
hu | Hungarian |
hy | Armenian |
id | Indonesian |
ig | Igbo |
is | Icelandic |
it | Italian |
ja | Japanese |
jv | Javanese |
ka | Georgian |
kk | Kazakh |
km | Khmer |
kn | Kannada |
ko | Korean |
ku | Kurdish |
ky | Kyrgyz |
La | Latin |
Lb | Luxembourgish |
Lo | Laothian |
Lt | Lithuanian |
Lv | Latvian |
mg | Malagasy |
mi | Maori |
mk | Macedonian |
ml | Malayalam |
mn | Mongolian |
mr | Marathi |
ms | Malay |
mt | Maltese |
my | Burmese |
ne | Nepali |
nl | Dutch |
no | Norwegian |
ny | Nyanja |
or | Oriya |
pa | Punjabi |
pl | Polish |
pt | Portuguese |
ro | Romanian |
ru | Russian |
rw | Kinyarwanda |
si | Sinhalese |
sk | Slovak |
sl | Slovenian |
sm | Samoan |
sn | Shona |
so | Somali |
sq | Albanian |
sr | Serbian |
st | Sesotho |
su | Sundanese |
sv | Swedish |
sw | Swahili |
ta | Tamil |
te | Telugu |
tg | Tajik |
th | Thai |
tk | Turkmen |
tl | Tagalog |
tr | Turkish |
tt | Tatar |
ug | Uighur |
uk | Ukrainian |
ur | Urdu |
uz | Uzbek |
vi | Vietnamese |
wo | Wolof |
xh | Xhosa |
yi | Yiddish |
yo | Yoruba |
zh | Chinese |
zu | Zulu |