The Classify Endpoint
In this chapter you'll learn how to classify a small dataset of sentences by their sentiment (positive, negative, or neutral), using Cohere's Classify endpoint
LLMs have been pre-trained with a vast amount of training data, allowing them to capture how words are being used and how their meaning changes depending on the context. A very common application of this is text classification. Cohere’s Classify endpoint makes it easy to take a list of texts and predict their categories, or classes. If you have read a little bit about text classification, you may have come across sentiment analysis, which is the task of classifying the sentiment of a text into a number of classes, say, positive, negative, or neutral. This is useful for applications like analyzing social media content or categorizing product feedback.
For example, a human can easily tell you that “Hello, World! What a beautiful day” conveys a positive sentiment, but let’s see if our models can do that too. And while we’re at it, let’s try classifying other phrases that you might find on social media.
This chapter comes with a corresponding Colab notebook, and we encourage you to follow it along as you read the chapter.
For the setup, please refer to the Setting Up chapter at the beginning of this module.
A typical machine learning model requires many training examples to perform text classification, but with the Classify endpoint, you can get started with as few as five examples per class. With the Classify endpoint, the input you need to prepare is as follows:
- These are the training examples we give the model to show the output we want it to generate.
- Each example contains the text itself and the corresponding label, or class.
- The minimum number of examples required is two per class.
- You can have as many classes as possible. If you are classifying text into two classes, that means you need a minimum of four examples, and if you have three, that means you need six examples, and so on.
- These are the list of text pieces you’d like to classify.
Our sentiment analysis classifier has three classes with five examples each: “Positive” for a positive sentiment, “Negative” for a negative sentiment, and “Neutral” for a neutral sentiment. The code looks as follows.
from cohere.responses.classify import Example examples = [Example("I'm so proud of you", "positive"), Example("What a great time to be alive", "positive"), Example("That's awesome work", "positive"), Example("The service was amazing", "positive"), Example("I love my family", "positive"), Example("They don't care about me", "negative"), Example("I hate this place", "negative"), Example("The most ridiculous thing I've ever heard", "negative"), Example("I am really frustrated", "negative"), Example("This is so unfair", "negative"), Example("This made me think", "neutral"), Example("The good old days", "neutral"), Example("What's the difference", "neutral"), Example("You can't ignore this", "neutral"), Example("That's how I see it", "neutral") ]
The inputs (we have twelve in this example):
inputs=["Hello, world! What a beautiful day", "It was a great time with great people", "Great place to work", "That was a wonderful evening", "Maybe this is why", "Let's start again", "That's how I see it", "These are all facts", "This is the worst thing", "I cannot stand this any longer", "This is really annoying", "I am just plain fed up" ]
With the Classify endpoint, setting up the model is quite straightforward. The main thing to do is to define the model type. For the Classify endpoint, we need to use an embedding model, which we'll use
embed-english-v2.0(we'll learn more about embedding models in the next chapter).
Putting everything together with the Classify endpoint looks like the following:
def classify_text(inputs, examples): response = co.classify( model='embed-english-v2.0', inputs=inputs, examples=examples) classifications = response.classifications return classifications predictions = classify_text(inputs,examples)
Together with the predicted class, the endpoint also returns the confidence value of the prediction (between 0 and 1). These confidence values are split among the classes, in this case three, in which the values add up to a total of 1. The classifier then selects the class with the highest confidence value as the “predicted class.” A high confidence value for the predicted class therefore indicates that the model is very confident of its prediction, and vice versa.
Here’s a sample output returned:
Input: Hello, world! What a beautiful day Prediction: positive Confidence: 0.83 ---------- Input: It was a great time with great people Prediction: positive Confidence: 0.99 ---------- Input: Great place to work Prediction: positive Confidence: 0.91 ---------- Input: That was a wonderful evening Prediction: positive Confidence: 0.96 ---------- Input: Maybe this is why Prediction: neutral Confidence: 0.70 ---------- Input: Let's start again Prediction: neutral Confidence: 0.83 ---------- Input: That's how I see it Prediction: neutral Confidence: 1.00 ---------- Input: These are all facts Prediction: neutral Confidence: 0.78 ---------- Input: This is the worst thing Prediction: negative Confidence: 0.93 ---------- Input: I cannot stand this any longer Prediction: negative Confidence: 0.93 ---------- Input: This is really annoying Prediction: negative Confidence: 0.99 ---------- Input: I am just plain fed up Prediction: negative Confidence: 1.00 ----------
The model returned a Positive sentiment for “Hello, world! What a beautiful day,” which is what we would expect! And the predictions for all the rest look spot on too.
That was one example, but you can classify any kind of text into any number of possible classes according to your needs.
As your task gets more complex, you will likely need to bring in additional training data and finetune a model. This will ensure that the model can capture the nuances specific to your task and realize performance gains. You can read more about finetuning representation models in one of the following chapters.
If you’d like to learn more about text classification, here are some additional resources:
- Intuition and use case examples
- An example application: toxicity detection
- Evaluating a classifier’s performance
- Classify endpoint API reference
In this chapter, you've learned to classify text based on mood using Cohere's
Classify endpoint. Follow along the next chapters to learn other endpoints and how to use them for many other tasks!
This material comes from the post Hello, World! Meet Language AI: Part 2
Updated 14 days ago
Now that you know how to classify, learn how to build embeddings with the Embed endpoint.