Building a Chatbot with Cohere

As its name implies, the Chat endpoint enables developers to build chatbots that can handle conversations. At the core of a conversation is a multi-turn dialog between the user and the chatbot. This requires the chatbot to have the state (or “memory”) of all the previous turns to maintain the state of the conversation.

In this tutorial, you’ll learn about:

• Creating a custom preamble
• Creating a single-turn conversation
• Building the conversation memory
• Running a multi-turn conversation
• Viewing the chat history

You’ll learn these by building an onboarding assistant for new hires.

Setup

To get started, first we need to install the cohere library and create a Cohere client.

1
# pip install cohere
2
3
import cohere
4
5
co = cohere.ClientV2(api_key"COHERE_API_KEY") # Get your free API key: https://dashboard.cohere.com/api-keys

Creating a custom preamble

A conversation starts with a system message, or a preamble, to help steer a chatbot’s response toward certain characteristics.

For example, if we want the chatbot to adopt a formal style, the preamble can be used to encourage the generation of more business-like and professional responses.

The recommended approach is to use two H2 Markdown headers: “Task and Context” and “Style Guide” in the exact order.

In the example below, the preamble provides context for the assistant’s task (task and context) and encourages the generation of rhymes as much as possible (style guide).

1
# Add the user message
2
message = "I'm joining a new startup called Co1t today. Could you help me write a short introduction message to my teammates."
3
4
# Create a custom system message
5
system_message="""## Task and Context
6
You are an assistant who assist new employees of Co1t with their first week.
7
8
## Style Guide
9
Try to speak in rhymes as much as possible. Be professional."""
10
11
# Add the messages
12
messages = [{"role": "system", "content": system_message},
13
            {"role": "user", "content": message}]
14
15
# Generate the response
16
response = co.chat(model="command-r-plus-08-2024",
17
                   messages=messages)
18
19
print(response.message.content[0].text)

Sure, here's a rhyme to break the ice,
A warm welcome to the team, so nice,
Hi, I'm [Your Name], a new face,
Ready to join the Co1t space,
A journey begins, a path unknown,
But together we'll make our mark, a foundation stone,
Excited to learn and contribute my part,
Let's create, innovate, and leave a lasting art,
Looking forward to our adventures yet untold,
With teamwork and passion, let's achieve our goals!
Cheers to a great start!
Your enthusiastic new mate.

Further reading:

• Documentation on preambles

Starting the first conversation turn

Let’s start with the first conversation turn.

Here, we are also adding a custom preamble or system message for generating a concise response, just to keep the outputs brief for this tutorial.

1
# Add the user message
2
message = "I'm joining a new startup called Co1t today. Could you help me write a short introduction message to my teammates."
3
4
# Create a custom system message
5
system_message="""## Task and Context
6
Generate concise responses, with maximum one-sentence."""
7
8
# Add the messages
9
messages = [{"role": "system", "content": system_message},
10
            {"role": "user", "content": message}]
11
12
# Generate the response
13
response = co.chat(model="command-r-plus-08-2024",
14
                   messages=messages)
15
16
print(response.message.content[0].text)

"Hello, teammates! I'm thrilled to join the Co1t family today and looking forward to getting to know you all and contributing to our shared success."

Building the conversation memory

Now, we want the model to refine the earlier response. This requires the next generation to have access to the state, or memory, of the conversation.

To do this, we append the messages with the model’s previous response using the assistant role.

Next, we also append a new user message (for the second turn) to the messages list.

Looking at the response, we see that the model is able to get the context from the chat history. The model is able to capture that “it” in the user message refers to the introduction message it had generated earlier.

1
# Append the previous response
2
messages.append({'role' : 'assistant', 'content': response.message.content[0].text})
3
4
# Add the user message
5
message = "Make it more upbeat and conversational."
6
7
# Append the user message
8
messages.append({"role": "user", "content": message})
9
10
# Generate the response with the current chat history as the context
11
response = co.chat(model="command-r-plus-08-2024",
12
                   messages=messages)
13
14
print(response.message.content[0].text)

"Hey, future Co1t buddies! Stoked to join this awesome team, let's get to know each other and make some startup magic together!"

Further reading:

• Documentation on using the Chat endpoint

Running a multi-turn conversation

You can continue doing this for any number of turns by continuing to append the chatbot’s response and the new user message to the messages list.

1
# Append the previous response
2
messages.append({"role": "assistant", "content": response.message.content[0].text})
3
4
# Add the user message
5
message = "Thanks. Could you create another one for my DM to my manager."
6
7
# Append the user message
8
messages.append({"role": "user", "content": message})
9
10
# Generate the response with the current chat history as the context
11
response = co.chat(model="command-r-plus-08-2024",
12
                   messages=messages)
13
14
print(response.message.content[0].text)

"Hi, boss! So excited to dive into my new role at Co1t and eager to learn from your mentorship and guidance. Let's crush it!"

Viewing the chat history

To look at the current chat history, you can print the messages list, which contains a list of user and assistant turns in the same sequence as they were created.

1
# Append the previous response
2
messages.append({"role": "assistant", "content": response.message.content[0].text})
3
4
# View the chat history
5
for message in messages:
6
    print(message,"\n")

{'role': 'system', 'content': '## Task and Context\nGenerate concise responses, with maximum one-sentence.'} 
{'role': 'user', 'content': "I'm joining a new startup called Co1t today. Could you help me write a short introduction message to my teammates."} 
{'role': 'assistant', 'content': '"Hello, teammates! I\'m thrilled to join the Co1t family today and looking forward to getting to know you all and contributing to our shared success."'} 
{'role': 'user', 'content': 'Make it more upbeat and conversational.'} 
{'role': 'assistant', 'content': '"Hey, future Co1t buddies! Stoked to join this awesome team, let\'s get to know each other and make some startup magic together!"'} 
{'role': 'user', 'content': 'Thanks. Could you create another one for my DM to my manager.'} 
{'role': 'assistant', 'content': '"Hi, boss! So excited to dive into my new role at Co1t and eager to learn from your mentorship and guidance. Let\'s crush it!"'} 

Conclusion

In this tutorial, you learned about:

• How to create a custom preamble
• How to create a single-turn conversation
• How to build the conversation memory
• How to run a multi-turn conversation
• How to view the chat history

You will use the same method for running a multi-turn conversation when you learn about other use cases such as RAG (Part 6) and tool use (Part 7).

But to fully leverage these other capabilities, you will need another type of language model that generates text representations, or embeddings.

In Part 4, you will learn how text embeddings can power an important use case for RAG, which is semantic search.