Retrieval Augmented Generation (RAG)

1
import cohere
2
co = cohere.ClientV2(api_key="<YOUR API KEY>")
3
4
# Retrieve the documents
5
documents = [
6
  {
7
    "data": {
8
      "title": "Tall penguins",
9
      "snippet": "Emperor penguins are the tallest."
10
    }
11
  },
12
  {
13
    "data": {
14
      "title": "Penguin habitats",
15
      "snippet": "Emperor penguins only live in Antarctica."
16
    }
17
  },
18
  {
19
    "data": {
20
      "title": "What are animals?",
21
      "snippet": "Animals are different from plants."
22
    }
23
  }
24
]
25
26
# Add the user message
27
message = "Where do the tallest penguins live?"
28
messages = [{"role": "user", "content": message}]
29
30
response = co.chat(
31
    model="command-r-plus-08-2024",
32
    messages=messages,
33
    documents=documents)
34
35
print(response.message.content[0].text)
36
37
print(response.message.citations)

# response.message.content[0].text
Emperor penguins are the tallest penguins. They only live in Antarctica.
# response.message.citations
[Citation(start=0,
          end=16, 
          text='Emperor penguins', 
          sources=[DocumentSource(type='document', id='doc:0', document={'id': 'doc:0', 'snippet': 'Emperor penguins are the tallest.', 'title': 'Tall penguins'})]), 
Citation(start=25, 
          end=42, 
          text='tallest penguins.', 
          sources=[DocumentSource(type='document', id='doc:0', document={'id': 'doc:0', 'snippet': 'Emperor penguins are the tallest.', 'title': 'Tall penguins'})]), 
Citation(start=61, 
          end=72, 
          text='Antarctica.',
          sources=[DocumentSource(type='document', id='doc:1', document={'id': 'doc:1', 'snippet': 'Emperor penguins only live in Antarctica.', 'title': 'Penguin habitats'})])]

# Sample response
['popularity of NSync', 'popularity of Backstreet Boys']

# Sample response
['NSync popularity', 'Backstreet Boys popularity', 'NSync vs Backstreet Boys popularity comparison', 'Which boy band is more popular NSync or Backstreet Boys', 'NSync and Backstreet Boys fan base size comparison', 'Who has sold more albums NSync or Backstreet Boys', 'NSync and Backstreet Boys chart performance comparison']

1
import cohere
2
co = cohere.ClientV2(api_key="<YOUR API KEY>")
3
4
documents = [
5
    {
6
        "data": {
7
            "title": "CSPC: Backstreet Boys Popularity Analysis - ChartMasters",
8
            "snippet": "↓ Skip to Main Content\n\nMusic industry – One step closer to being accurate\n\nCSPC: Backstreet Boys Popularity Analysis\n\nHernán Lopez Posted on February 9, 2017 Posted in CSPC 72 Comments Tagged with Backstreet Boys, Boy band\n\nAt one point, Backstreet Boys defined success: massive albums sales across the globe, great singles sales, plenty of chart topping releases, hugely hyped tours and tremendous media coverage.\n\nIt is true that they benefited from extraordinarily good market conditions in all markets. After all, the all-time record year for the music business, as far as revenues in billion dollars are concerned, was actually 1999. That is, back when this five men group was at its peak.",
9
        }
10
    },
11
    {
12
        "data": {
13
            "title": "CSPC: NSYNC Popularity Analysis - ChartMasters",
14
            "snippet": "↓ Skip to Main Content\n\nMusic industry – One step closer to being accurate\n\nCSPC: NSYNC Popularity Analysis\n\nMJD Posted on February 9, 2018 Posted in CSPC 27 Comments Tagged with Boy band, N'Sync\n\nAt the turn of the millennium three teen acts were huge in the US, the Backstreet Boys, Britney Spears and NSYNC. The latter is the only one we haven’t study so far. It took 15 years and Adele to break their record of 2,4 million units sold of No Strings Attached in its first week alone.\n\nIt wasn’t a fluke, as the second fastest selling album of the Soundscan era prior 2015, was also theirs since Celebrity debuted with 1,88 million units sold.",
15
        }
16
    },
17
    {
18
        "data": {
19
            "title": "CSPC: Backstreet Boys Popularity Analysis - ChartMasters",
20
            "snippet": " 1997, 1998, 2000 and 2001 also rank amongst some of the very best years.\n\nYet the way many music consumers – especially teenagers and young women’s – embraced their output deserves its own chapter. If Jonas Brothers and more recently One Direction reached a great level of popularity during the past decade, the type of success achieved by Backstreet Boys is in a completely different level as they really dominated the business for a few years all over the world, including in some countries that were traditionally hard to penetrate for Western artists.\n\nWe will try to analyze the extent of that hegemony with this new article with final results which will more than surprise many readers.",
21
        }
22
    },
23
    {
24
        "data": {
25
            "title": "CSPC: NSYNC Popularity Analysis - ChartMasters",
26
            "snippet": " Was the teen group led by Justin Timberlake really that big? Was it only in the US where they found success? Or were they a global phenomenon?\n\nAs usual, I’ll be using the Commensurate Sales to Popularity Concept in order to relevantly gauge their results. This concept will not only bring you sales information for all NSYNC‘s albums, physical and download singles, as well as audio and video streaming, but it will also determine their true popularity. If you are not yet familiar with the CSPC method, the next page explains it with a short video. I fully recommend watching the video before getting into the sales figures.",
27
        }
28
    },
29
]
30
31
# Add the user message
32
message = "Who is more popular: Nsync or Backstreet Boys?"
33
messages = [{"role": "user", "content": message}]
34
35
response = co.chat(
36
    model="command-r-plus-08-2024",
37
    messages=messages,
38
    documents=documents
39
)
40
41
print(response.message.content[0].text)
42
43
print(response.message.citations)

# response.message.content[0].text
Both NSYNC and Backstreet Boys were huge in the US at the turn of the millennium. However, Backstreet Boys achieved a greater level of success than NSYNC. They dominated the music business for a few years all over the world, including in some countries that were traditionally hard to penetrate for Western artists. Their success included massive album sales across the globe, great singles sales, plenty of chart-topping releases, hugely hyped tours and tremendous media coverage.
# response.message.citations (truncated for brevity)
[Citation(start=36, 
          end=81, 
          text='huge in the US at the turn of the millennium.', 
          sources=[DocumentSource(type='document', id='doc:1', document={'id': 'doc:1', 'snippet': "↓ Skip to Main Content\n\nMusic industry – One step closer ...", 'title': 'CSPC: NSYNC Popularity Analysis - ChartMasters'})]),
Citation(start=107, 
          end=154, 
          text='achieved a greater level of success than NSYNC.', 
          sources=[DocumentSource(type='document', id='doc:2', document={'id': 'doc:2', 'snippet': ' 1997, 1998, 2000 and 2001 also rank amongst some of the very best ...', 'title': 'CSPC: Backstreet Boys Popularity Analysis - ChartMasters'})]),
Citation(start=160, 
        end=223,
        ...
...]