Querying Structured Data (Tables)

Open in Colab

In the previous tutorials, we explored how to build agentic RAG applications over unstructured and semi-structured data. In this tutorial and the next, we’ll turn our focus to structured data.

This tutorial focuses on querying tables, and the next tutorial will be about querying SQL databases.

Consider a scenario where you have a CSV file containing evaluation results for an LLM application.

A user might ask questions like “What’s the average score for a specific use case?” or “Which configuration has the lowest latency?”. These queries require not just retrieval, but also data analysis and interpretation.

In this tutorial, we’ll cover:

Creating a function to execute Python code
Setting up a tool to interact with tabular data
Building an agent for querying tabular data
Running the agent

Let’s get started by setting up our environment and defining the necessary tools for our agent.

Setup

To get started, first we need to install the cohere library and create a Cohere client.

PYTHON

! pip install cohere pandas -qq

PYTHON

import json
import os
import cohere
co = cohere.ClientV2(
    "COHERE_API_KEY"
)  # Get your free API key: https://dashboard.cohere.com/api-keys

And here’s the data we’ll be working with. evaluation_results.csv is a CSV file containing evaluation results for a set of LLM applications - name extraction, email drafting, and article summarization.

The file has the following columns:

usecase: The use case.
run: The run ID.
score: The evaluation score for a particular run.
temperature: The temperature setting of the model for a particular run.
tokens: The number of tokens generated by the model for a particular run.
latency: The latency of the model’s response for a particular run.

Important: the data can be found here. Make sure to have the file in the same directory as this notebook for the imports to work correctly.

PYTHON

import pandas as pd
df = pd.read_csv("evaluation_results.csv")
df.head()

	usecase	run	score	temperature	tokens	latency
0	extract_names	A	0.5	0.3	103	1.12
1	draft_email	A	0.6	0.3	252	2.50
2	summarize_article	A	0.8	0.3	350	4.20
3	extract_names	B	0.2	0.3	101	2.85
4	draft_email	B	0.4	0.3	230	3.20

Creating a function to execute Python code

Here, we introduce a new tool that allows the agent to execute Python code and return the result. The agent will use this tool to generate pandas code for querying the data.

To create this tool, we’ll use the PythonREPL class from the langchain_experimental.utilities module. This class provides a sandboxed environment for executing Python code and returns the result.

First, we define a python_tool that uses the PythonREPL class to execute Python code and return the result.

Next, we define a ToolInput class to handle the input for the python_tool.

Finally, we create a function analyze_evaluation_results that takes a string of Python code as input, executes the code using the Python tool we created, and returns the result.

IMPORTANT:

The source code for tool definitions can be found here. Make sure to have the tool_def.py file in the same directory as this notebook for the imports to work correctly.

PYTHON

from tool_def import (
    analyze_evaluation_results,
    analyze_evaluation_results_tool,
)

PYTHON

functions_map = {
    "analyze_evaluation_results": analyze_evaluation_results
}

Setting up a tool to interact with tabular data

Next, we define the analyze_evaluation_results tool. There are many ways we can set up a tool to work with CSV data, and in this example, we are using the tool description to provide the agent with the necessary context for working with the CSV file, such as:

the name of the CSV file to load
the columns of the CSV file
additional instructions on what libraries to use (in this case, pandas)

The parameter of this tool is the code string containing the Python code that the agent writes to analyze the data.

PYTHON

analyze_evaluation_results_tool = {
    "type": "function",
    "function": {
        "name": "analyze_evaluation_results",
        "description": "Generate Python code using the pandas library to analyze evaluation results from a dataframe called `evaluation_results`. The dataframe has columns 'usecase','run','score','temperature','tokens', and 'latency'. You must start with `import pandas as pd` and read a CSV file called `evaluation_results.csv` into the `evaluation_results` dataframe.",
        "parameters": {
            "type": "object",
            "properties": {
                "code": {
                    "type": "string",
                    "description": "Executable Python code",
                }
            },
            "required": ["code"],
        },
    },
}

PYTHON

tools = [analyze_evaluation_results_tool]

Building an agent for querying tabular data

Next, let’s create a run_agent function to run the agentic RAG workflow, the same as in Part 1.

The only change we are making here is to make the system message simpler and more specific since the agent now only has one tool.

PYTHON

system_message = """## Task and Context
ou are an assistant who helps developers analyze LLM application evaluation results from a CSV files."""

PYTHON

model = "command-a-plus-05-2026"
def run_agent(query, messages=None):
    if messages is None:
        messages = []
    if "system" not in {m.get("role") for m in messages}:
        messages.append({"role": "system", "content": system_message})
    # Step 1: get user message
    print(f"Question:\n{query}")
    print("=" * 50)
    messages.append({"role": "user", "content": query})
    # Step 2: Generate tool calls (if any)
    response = co.chat(
        model=model, messages=messages, tools=tools, temperature=0.3
    )
    while response.message.tool_calls:
        print("TOOL PLAN:")
        print(response.message.tool_plan, "\n")
        print("TOOL CALLS:")
        for tc in response.message.tool_calls:
            if tc.function.name == "analyze_evaluation_results":
                print(f"Tool name: {tc.function.name}")
                tool_call_prettified = print(
                    "\n".join(
                        f"  {line}"
                        for line_num, line in enumerate(
                            json.loads(tc.function.arguments)[
                                "code"
                            ].splitlines()
                        )
                    )
                )
                print(tool_call_prettified)
            else:
                print(
                    f"Tool name: {tc.function.name} | Parameters: {tc.function.arguments}"
                )
        print("=" * 50)
        messages.append(response.message)
        # Step 3: Get tool results
        for tc in response.message.tool_calls:
            tool_result = functions_map[tc.function.name](
                **json.loads(tc.function.arguments)
            )
            tool_content = [
                {
                    "type": "document",
                    "document": {"data": json.dumps(tool_result)},
                }
            ]
            messages.append(
                {
                    "role": "tool",
                    "tool_call_id": tc.id,
                    "content": tool_content,
                }
            )
        # Step 4: Generate response and citations
        response = co.chat(
            model=model,
            messages=messages,
            tools=tools,
            temperature=0.3,
        )
    messages.append(
        {
            "role": "assistant",
            "content": response.message.content[0].text,
        }
    )
    # Print final response
    print("RESPONSE:")
    print(response.message.content[0].text)
    print("=" * 50)
    # Print citations (if any)
    verbose_source = (
        False  # Change to True to display the contents of a source
    )
    if response.message.citations:
        print("CITATIONS:\n")
        for citation in response.message.citations:
            print(
                f"Start: {citation.start}| End:{citation.end}| Text:'{citation.text}' "
            )
            print("Sources:")
            for idx, source in enumerate(citation.sources):
                print(f"{idx+1}. {source.id}")
                if verbose_source:
                    print(f"{source.tool_output}")
            print("\n")
    return messages

Running the agent

Let’s ask the agent a few questions, starting with this query about the average evaluation score in run A.

To answer this query, the agent needs to write Python code that uses the pandas library to calculate the average evaluation score in run A. And it gets the answer right.

PYTHON

messages = run_agent("What's the average evaluation score in run A")
# Answer: 0.63

Question:
What's the average evaluation score in run A
==================================================
Python REPL can execute arbitrary code. Use with caution.
TOOL PLAN:
I will write and execute Python code to calculate the average evaluation score in run A. 
TOOL CALLS:
Tool name: analyze_evaluation_results
    import pandas as pd
    
    df = pd.read_csv("evaluation_results.csv")
    
    # Calculate the average evaluation score in run A
    average_score_run_A = df[df["run"] == "A"]["score"].mean()
    
    print(f"Average evaluation score in run A: {average_score_run_A}")
None
==================================================
RESPONSE:
The average evaluation score in run A is 0.63.
==================================================
CITATIONS:
Start: 41| End:46| Text:'0.63.' 
Sources:
1. analyze_evaluation_results_phqpwwat2hgf:0

Next, we ask a slightly more complex question, this time about the latency of the highest-scoring run for one use case. This requires the agent to filter based on the use case, find the highest-scoring run, and return the latency value.

PYTHON

messages = run_agent(
    "What's the latency of the highest-scoring run for the summarize_article use case?"
)
# Answer: 4.8

Question:
What's the latency of the highest-scoring run for the summarize_article use case?
==================================================
TOOL PLAN:
I will write Python code to find the latency of the highest-scoring run for the summarize_article use case. 
TOOL CALLS:
Tool name: analyze_evaluation_results
    import pandas as pd
    
    df = pd.read_csv("evaluation_results.csv")
    
    # Filter for the summarize_article use case
    use_case_df = df[df["usecase"] == "summarize_article"]
    
    # Find the highest-scoring run
    highest_score_run = use_case_df.loc[use_case_df["score"].idxmax()]
    
    # Get the latency of the highest-scoring run
    latency = highest_score_run["latency"]
    
    print(f"Latency of the highest-scoring run: {latency}")
None
==================================================
RESPONSE:
The latency of the highest-scoring run for the summarize_article use case is 4.8.
==================================================
CITATIONS:
Start: 77| End:81| Text:'4.8.' 
Sources:
1. analyze_evaluation_results_es3hnnnp5pey:0

Next, we ask a question to compare the use cases in terms of token usage, and to show a markdown table to show the comparison.

PYTHON

messages = run_agent(
    "Which use case uses the least amount of tokens on average? Show the comparison of all use cases in a markdown table."
)
# Answer: extract_names (106.25), draft_email (245.75), summarize_article (355.75)

Question:
Which use case uses the least amount of tokens on average? Show the comparison of all use cases in a markdown table.
==================================================
TOOL PLAN:
I will use the analyze_evaluation_results tool to generate Python code to find the use case that uses the least amount of tokens on average. I will also generate code to create a markdown table to compare all use cases. 
TOOL CALLS:
Tool name: analyze_evaluation_results
    import pandas as pd
    
    evaluation_results = pd.read_csv("evaluation_results.csv")
    
    # Group by 'usecase' and calculate the average tokens
    avg_tokens_by_usecase = evaluation_results.groupby('usecase')['tokens'].mean()
    
    # Find the use case with the least average tokens
    least_avg_tokens_usecase = avg_tokens_by_usecase.idxmin()
    
    print(f"Use case with the least average tokens: {least_avg_tokens_usecase}")
    
    # Create a markdown table comparing average tokens for all use cases
    markdown_table = avg_tokens_by_usecase.reset_index()
    markdown_table.columns = ["Use Case", "Average Tokens"]
    print(markdown_table.to_markdown(index=False))
None
==================================================
RESPONSE:
The use case that uses the least amount of tokens on average is extract_names.
Here is a markdown table comparing the average tokens for all use cases:
| Use Case | Average Tokens |
|:-------------------------|-------------------------------:|
| draft_email | 245.75 |
| extract_names | 106.25 |
| summarize_article | 355.75 |
==================================================
CITATIONS:
Start: 64| End:78| Text:'extract_names.' 
Sources:
1. analyze_evaluation_results_zp68h5304e3v:0
Start: 156| End:164| Text:'Use Case' 
Sources:
1. analyze_evaluation_results_zp68h5304e3v:0
Start: 167| End:181| Text:'Average Tokens' 
Sources:
1. analyze_evaluation_results_zp68h5304e3v:0
Start: 248| End:259| Text:'draft_email' 
Sources:
1. analyze_evaluation_results_zp68h5304e3v:0
Start: 262| End:268| Text:'245.75' 
Sources:
1. analyze_evaluation_results_zp68h5304e3v:0
Start: 273| End:286| Text:'extract_names' 
Sources:
1. analyze_evaluation_results_zp68h5304e3v:0
Start: 289| End:295| Text:'106.25' 
Sources:
1. analyze_evaluation_results_zp68h5304e3v:0
Start: 300| End:317| Text:'summarize_article' 
Sources:
1. analyze_evaluation_results_zp68h5304e3v:0
Start: 320| End:326| Text:'355.75' 
Sources:
1. analyze_evaluation_results_zp68h5304e3v:0

Summary

In this tutorial, we learned about:

How to create a function to execute Python code
How to set up a tool to interact with tabular data
How to run the agent

By implementing these techniques, we’ve expanded our agentic RAG system to handle structured data in the form of tables.

While this tutorial demonstrated how to work with tabular data using pandas and Python, the agentic RAG approach can be applied to other forms of structured data as well. This means we can build agents that can translate natural language queries into various types of data analysis tasks.

In Part 6, we’ll learn how to do structured query generation for SQL databases.