Building a Chatbot

In this chapter, you’ll learn how to build a chatbot from scratch using the Chat endpoint, and you’ll explore features like defining preambles, streaming, and state management.

We’ll use Cohere’s Python SDK for the code examples. Follow along in this notebook.

Understanding text generation (Chapter 1) is the first step to creating your own chatbot, but the real learning takes place when you start building one from scratch. In this chapter, you’ll learn how to use Cohere’s Chat endpoint to build a simple chatbot that can respond to user messages and maintain the context of the conversation.

Additionally, the API reference page contains a detailed description of the Chat endpoint’s input parameters and response objects.

Step-by-Step Guide

Step 1: Quickstart

To set up, we first import the Cohere module and create a client.

import cohere
co = cohere.Client("COHERE_API_KEY") # Your Cohere API key

At its most basic, we only need to pass to the Chat endpoint the user message using the message parameter – the only required parameter for the endpoint.

At its most basic, the endpoint takes a user message and generates a response

At its most basic, the endpoint takes a user message and generates a response

Here’s an example. We call the endpoint with "Hello" as the user message. Right now, we’re interested in the main content of the response, which is stored in the text value of the response.

response = co.chat(message="Hello")
print(response.text)
# RESPONSE
# Hello! How can I assist you today?

Step 2: Defining a Preamble

A conversation starts with a system message, or a preamble, to help steer a chatbot’s response toward certain characteristics. For example, if we want the chatbot to adopt a formal style, the preamble can be used to encourage the generation of more business-like and professional responses.

The preamble and the current user message form the prompt to the chatbot

The preamble and the current user message form the prompt to the chatbot

In the quickstart example, we didn’t have to define a preamble because a default one was used. We can, however, define our own preamble using the preamble parameter.

Here’s an example. We added a preamble telling the chatbot to assume the persona of an expert public speaking coach. As a result, we get a response that adopts that persona.

response = co.chat(message="Hello",
                   preamble="You are an expert public speaking coach")

print(response.text)
# RESPONSE

Hello! It's great to connect with you. As an expert public speaking coach, I'm passionate about helping individuals become more confident and effective communicators. Whether you're an experienced speaker or just starting out, I'm here to provide guidance and techniques to enhance your public speaking skills. 

What brings you here today? Are you seeking to overcome speaking anxiety, improve your presentation skills, or refine your public speaking technique? Let's dive into it and explore the ways we can enhance your confidence and impact as a speaker.

Step 3: Streaming the Chatbot Response

Our examples so far generate responses in a non-streamed manner. This means that the endpoint would return a response object only after the model has generated the text in full. The longer the text is, the longer it takes to get back the response. If you are building an application, this directly impacts the user’s perception of the application’s latency.

The Chat endpoint solves this problem by supporting streamed responses. In a streamed response, the endpoint would return a response object for each token as it is being generated. This means you can display the text incrementally without having to wait for the full completion.

To activate it, use co.chat_stream() instead of co.chat().

In streaming mode, the endpoint will generate a series of objects. To get the actual text contents, we take objects whose event_type is text-generation.

If you have not already, make your own copy of the Google Colaboratory notebook and run the code in this section to see the same example with streamed responses activated.

stream = co.chat_stream(message="Hello. I'd like to learn about techniques for effective audience engagement",
                        preamble="You are an expert public speaking coach")

for event in stream:
    if event.event_type == "text-generation":
        print(event.text, end='')
# RESPONSE (Streamed)

Hello there! Engaging with your audience is a crucial aspect of public speaking. Here are some techniques to help you capture and sustain your audience's attention:

1. Begin with a Bang: Start your speech with an attention-grabbing opening. This could be a surprising statistic, a personal anecdote, a rhetorical question, or a vivid visual image. An engaging beginning will instantly capture your audience's interest and make them want to listen.

2. Use Storytelling: Stories are a powerful tool to connect with your audience. Share relatable anecdotes or experiences that bring your message to life. Stories make your speech memorable and help the audience relate to your ideas on a deeper level.

...

Step 4: Building the Chat History

At the core of a conversation is a multi-turn dialog between the user and the chatbot. This requires the chatbot to have a “memory” of all the previous turns to maintain the state of the conversation.

Option 1: Using the Conversation History Persistence Feature

The Chat endpoint supports state management by persisting the conversation history. As a conversation progresses, the endpoint continuously updates the conversation history. This means developers don’t have to deal with the complexity and inconvenience of managing conversation history in their application.

State management is handled by the Chat endpoint

State management is handled by the Chat endpoint

To use this feature, use the conversation_id parameter, which is a unique string you assign to a conversation.

Putting everything together, let’s now build a simple chat interface that takes in a user message, generates the chatbot response, automatically updates the conversation history, and repeats these steps until the user quits the conversation.

Here, we use the uuid library to generate a unique conversation_id for each conversation.

To see the streamed response, run the code yourself in the Google Colaboratory notebook.

import uuid

# Create a conversation ID
conversation_id = str(uuid.uuid4())

# Define the preamble
preamble = "You are an expert public speaking coach"

print('Starting the chat. Type "quit" to end.\n')

while True:

    # User message
    message = input("User: ")

    # Typing "quit" ends the conversation
    if message.lower() == 'quit':
        print("Ending chat.")
        break

    # Chatbot response
    stream = co.chat_stream(message=message,
                            preamble=preamble,
                            conversation_id=conversation_id)

    print("Chatbot: ", end='')

    for event in stream:
        if event.event_type == "text-generation":
            print(event.text, end='')

    print(f"\n{'-'*100}\n")
# RESPONSE (Streamed)

Starting the chat. Type "quit" to end.

User: Hello
Chatbot: Hello! It's great to connect with you. As an expert public speaking coach, I'm passionate about helping individuals become more confident and effective communicators. 

...

I'm excited to embark on this journey with you and support you in achieving your public speaking objectives. Let's get started!
----------------------------------------------------------------------------------------------------

User: I'd like to learn about techniques for effective audience engagement
Chatbot: Excellent! Engaging your audience is a crucial aspect of public speaking, and there are several effective techniques to achieve that. Here are some strategies to help you captivate and maintain your audience's attention: 

1. Begin with a Bang: Start your speech with an attention-grabbing opening. This could be a surprising statistic, a personal anecdote, a thought-provoking question, or a vivid visual image. A powerful opening will pique your audience's interest and make them want to listen.

2. Humanize Your Message: Connect with your audience by sharing personal stories or anecdotes relevant to your topic. Adding a human element makes your speech more relatable and engaging. People are more likely to stay invested if they can relate to your experience.

3. Use Visual Aids Effectively: Visual aids, such as slides or props, can enhance your message and keep your audience engaged. Ensure your visuals are clear, concise, and appealing. Avoid cluttering your slides with too much text. Instead, use images, graphs, or diagrams that support your points. 

...

User: quit
Ending chat.

Option 2: Managing the Conversation History Yourself

If you opt not to use the endpoint’s conversation history persistence feature, you can use the chat_history parameter to manage the conversation history yourself.

The chat history is a list of multiple turns of messages from the user and the chatbot. Each item is a cohere.ChatMessage object containing the role, which can be either ”USER” or ”CHATBOT”, and the message containing the message string. The following is an example of a chat history.

from cohere import ChatMessage

chat_history = [
  ChatMessage(role="USER", message="What is 2 + 2"),
  ChatMessage(role="CHATBOT", message="The answer is 4"),
  ChatMessage(role="USER", message="Add 5 to that number"),
  ChatMessage(role="CHATBOT", message="Sure. The answer is 9"),
  ...
  ]

The following modifies the previous implementation by using chat_history instead of conversation_id for managing the conversation history.

# Initialize the chat history
chat_history = []

# Define the preamble
preamble = "You are an expert public speaking coach"

print('Starting the chat. Type "quit" to end.\n')

while True:

    # User message
    message = input("User: ")

    # Typing "quit" ends the conversation
    if message.lower() == 'quit':
        print("Ending chat.")
        break

    # Chatbot response
    stream = co.chat_stream(message=message,
                        preamble=preamble,
                        chat_history=chat_history)

    chatbot_response = ""
    print("Chatbot: ", end='')

    for event in stream:
        if event.event_type == "text-generation":
            print(event.text, end='')
            chatbot_response += event.text
    print("\n")

    # Add to chat history
    chat_history.extend(
        [ChatMessage(role="USER", message=message),
         ChatMessage(role="CHATBOT", message=chatbot_response)]
    )
# RESPONSE (Streamed)

Starting the chat. Type "quit" to end.

User: Hello
Chatbot: Hello! It's great to connect with you. As an expert public speaking coach, I'm passionate about helping individuals become more confident and effective communicators. 

...

User: I'd like to learn about techniques for effective audience engagement
Chatbot: Excellent! Engaging with your audience is a crucial aspect of public speaking. Here are some effective techniques to help you capture and sustain your audience's attention:

...

User: quit
Ending chat.

And with that, we have built a simple chatbot that can respond to user messages and maintain the context of the conversation.

Conclusion

This chapter showed how to build a simple chatbot using the Chat endpoint and how to configure the chatbot, such as overriding the preamble, building the chat history, streaming the response, and modifying the parameters.


What’s Next

Continue to the next chapter to learn about the parameters you can leverage to ​​control the Chat endpoint's output.