Agent API Calls

Marco Del TrediciMarco Del Tredici
Back to CookbooksOpen in GitHub

Motivation

Agents can play a crucial role in different enterprise scenarios. One such example is creating a request to fetch results from an existing enterprise API with specific requirements. For example, given a user query:

Retrieve id 7410e652-639d-402e-984e-8fd7025f0aac 8bb21b93-2ddf-4a63-af63-ddb6b1be49a1, ref GLPNT0005GUJOGGE GLSBR000BGASOBRE, nmg 0000234GLCHL0200ARTINDIUS'

The expected API request is:

[{"uuid":["7410e652-639d-402e-984e-8fd7025f0aac","8bb21b93-2ddf-4a63-af63-ddb6b1be49a1"],"page":0,"size":10}, {"objref":["GLPNT0005GUJOGGE","GLSBR000BGASOBRE"],"page":0,"size":10},{"nmgs":["0000234GLCHL0200ARTINDIUS"],"page":0,"size":10}]

This kind of task poses a major challenge, namely, the model must be very precise at identifying the codes in the query, based on their alphanumeric patterns, and matching these codes with the right parameter - for example, the code GLSBR000BGASOBRE needs to map to objref. The task is even more challenging when multiple APIs are available, and the model has to deal with a plethora of parameters.

Solution

In this notebook, we propose a simple but effective solution to the challenge defined above: We create a Langchain ReAct Agent that has access to a deterministic tool that extracts the alphanumeric patterns in the query, and returns a dictionary in which the keys are the parameters, and the values the extracted patterns. The output of the tool is then used to generate the final request.

Using such a tool is just one of the possibilities that the Agent has to generate the query. As we will see below, when a more semantic understanding of the query is required, we can ignore the tool and leverage the linguistic capabilities of the LLM powering the Agent to generate the final output.

With this approach, we bring together the best of two worlds: the ability of LLMs to use tools and generate outputs and the reliability and efficiency of deterministic functions.

Step 1: Setup

PYTHON
1# Uncomment if you need to install the following packages
2# !pip install cohere
3# !pip install python-dotenv
4# !pip install pandas
PYTHON
1import os
2import json
3import re
4import getpass
5from langchain.agents import AgentExecutor
6from langchain_cohere.chat_models import ChatCohere
7from langchain_cohere.react_multi_hop.agent import create_cohere_react_agent
8from langchain_core.prompts import ChatPromptTemplate
9from langchain_core.pydantic_v1 import BaseModel, Field
10from langchain_core.tools import tool
PYTHON
1# load the cohere api key
2os.environ["COHERE_API_KEY"] = getpass.getpass()

Step 2: Define the Tool and the Agent

Here we create a tool which implements the deterministic function to extract alphanumeric strings from the user’s query and match them to the right parameter.

PYTHON
1@tool
2def regex_extractor(user_query: str) -> dict:
3    """Function which, given the query from the user, returns a dictionary parameter:value."""
4    uuid = re.findall("\s([a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12})", user_query)
5    nmgs = re.findall("(0000[A-Z0-9]{21})", user_query)
6    objref = re.findall("\s([A-Z]{5,9}\d{3,4}[A-Z]{3,8})", user_query)
7    urn = re.findall("urn:[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}",user_query,)
8    d = {"uuid": uuid,
9         "nmgs": nmgs,
10         "objref": objref,
11         "urn": urn}
12    d_filtered = {k: v for k, v in d.items() if v != []}
13    return d_filtered
14
15class extract_code_v1productssearch(BaseModel):
16    user_query: str = Field(
17        description="This is the full input query as received from the user. Do not truncate or modify the query in any way."
18    )
19regex_extractor.name = "regex_extractor"
20regex_extractor.args_schema = extract_code_v1productssearch
21tools=[regex_extractor]
PYTHON
1# Let's define the preamble for the Agent.
2# The preamble includes info about:
3# - the tool the Agent has access to
4# - the cases in which the Agent has to produce an output without using the tool
5# - some examples to clarify the task
6preamble = """You are an assistant that given a user's query about, generates a request an API.
7You can use the tool named "regex_extractor".
8Pass to the "regex_extractor" tool the entire text of the the input query.
9The tool returns a dictionary, in which keys are the name of the codes, and values a list of extracted codes.
10Create a JSON for each of the key-value pairs in the dictionary.
11
12Return a list of JSONs. Make sure each JSON is properly defined. Do not return any other explanation or comment.
13You MUST generate a perfect JSON object: make sure that each string in the lists is included between quotes.
14
15If the request mentions one of the tags listed below, or a related word, create a dictionary in which the key is "taxonomies" and the value the list of capitalized tags.
16Tags list: cars, trucks, clothes, sport
17
18
19Find a list of examples here:
20User question | parameter for the tool | what you should understand
21Look products GLCMS004AGTCAMIS; 0000234GLCMS0100ANORAKCAA, GLCHL000CGUCHALE | Look products GLCMS004AGTCAMIS; 0000234GLCMS0100ANORAKCAA, GLCHL000CGUCHALE | [{"objref":["GLCMS004AGTCAMIS","GLCHL000CGUCHALE"]},{"nmgs":["0000234GLCMS0100ANORAKCAA"]}]
22Retrieve id 7410e652-639d-402e-984e-8fd7025f0aac 8bb21b93-2ddf-4a63-af63-ddb6b1be49a1, ref GLPNT0005GUJOGGE GLSBR000BGASOBRE, nmg 0000234GLCHL0200ARTINDIUS | Retrieve id 7410e652-639d-402e-984e-8fd7025f0aac 8bb21b93-2ddf-4a63-af63-ddb6b1be49a1, ref GLPNT0005GUJOGGE GLSBR000BGASOBRE, nmg 0000234GLCHL0200ARTINDIUS | [{"uuid":["7410e652-639d-402e-984e-8fd7025f0aac","8bb21b93-2ddf-4a63-af63-ddb6b1be49a1"]}, {"objref":["GLPNT0005GUJOGGE","GLSBR000BGASOBRE"]},{"nmgs":["0000234GLCHL0200ARTINDIUS"]}]
23Look for items of cars and trucks | Look for items of pants and t-shirts | [{'taxonomies': ['CARS', 'TRUCKS']}]
24Search products sport | Search products dress and jumpsuit | [{'taxonomies': ['SPORT']}]
25"""
PYTHON
1# Define the prompt
2prompt = ChatPromptTemplate.from_template("{input}")
3# Define the agent
4llm = ChatCohere(model="command-r-plus", temperature=0)
5# instantiate agent and agent executor
6agent = create_cohere_react_agent(
7   llm=llm,
8   tools=tools,
9   prompt=prompt,
10)
11agent_executor = AgentExecutor(agent=agent,
12                               tools=tools,
13                               verbose=True,
14                               return_intermediate_steps=True
15                    )
PYTHON
1# finally, let's write a function to convert the Agents output to a json
2def convert_to_json(string: str) -> json:
3    return json.loads(
4                string.replace("\xa0", " ")
5                .replace("json", "")
6                .replace("`", "")
7                .replace("`", "")
8            )

Step 3: Run the Agent

Let’s now test the Agent we just defined!

PYTHON
1query_1 = "Look for urn:75f2b737-06dd-4399-9206-a6c11b65138e, GLCMS004AGTCAMIS; 0000234GLCMS0100ANORAKCAA, GLCHL000CGUCHALE"
2response_1 = agent_executor.invoke(
3            {
4                "input": query_1,
5                "preamble": preamble,
6            }
7        )
Output
> Entering new AgentExecutor chain...

I will use the regex_extractor tool to extract the codes from the user query.
{'tool_name': 'regex_extractor', 'parameters': {'user_query': 'Look for urn:75f2b737-06dd-4399-9206-a6c11b65138e, GLCMS004AGTCAMIS; 0000234GLCMS0100ANORAKCAA, GLCHL000CGUCHALE'}}
{'nmgs': ['0000234GLCMS0100ANORAKCAA'], 'objref': ['GLCMS004AGTCAMIS', 'GLCHL000CGUCHALE'], 'urn': ['urn:75f2b737-06dd-4399-9206-a6c11b65138e']}Relevant Documents: 0
Cited Documents: 0
Answer: ```json JSON
[
    {
        "urn": ["urn:75f2b737-06dd-4399-9206-a6c11b65138e"],
        "objref": ["GLCMS004AGTCAMIS", "GLCHL000CGUCHALE"],
        "nmgs": ["0000234GLCMS0100ANORAKCAA"]
    }
]
```
Grounded answer: ```json JSON
  [
    {
        "urn": [<co: 0>"urn:75f2b737-06dd-4399-9206-a6c11b65138e</co: 0>"],
        "objref": [<co: 0>"GLCMS004AGTCAMIS</co: 0>", <co: 0>"GLCHL000CGUCHALE</co: 0>"],
        "nmgs": [<co: 0>"0000234GLCMS0100ANORAKCAA</co: 0>"]
    }
]
```
> Finished chain.

In the reasoning chain above, we can see that the Agent uses the tool we provided it to extract the strings in the query. The output of the tool is then used to generate the request.

PYTHON
1# let's have a look at the final output
2convert_to_json(response_1['output'])
Output
1[{'urn': ['urn:75f2b737-06dd-4399-9206-a6c11b65138e'],
2  'objref': ['GLCMS004AGTCAMIS', 'GLCHL000CGUCHALE'],
3  'nmgs': ['0000234GLCMS0100ANORAKCAA']}]

As mentioned above, the Agent can use the tool when specific alphanumeric patterns have to be extracted from the query; however, it can also generate the output based on its semantic understanding of the query. For example:

PYTHON
1query_2 = "I need tennis products"
2
3response_2 = agent_executor.invoke(
4    {
5        "input": query_2,
6        "preamble": preamble,
7    }
8)
Output
> Entering new AgentExecutor chain...

I will use the regex_extractor tool to extract the relevant information from the user request.
{'tool_name': 'regex_extractor', 'parameters': {'user_query': 'I need tennis products'}}
{}Relevant Documents: None
Cited Documents: None
Answer: ```json JSON
[
    {
        "taxonomies": [
            "SPORT"
        ]
    }
]
```
Grounded answer: ```json JSON
  [
  {
  "taxonomies": [
  "SPORT"
  ]
  }
]
```
> Finished chain.

The Agent runs the tool to check if any target string was in the query, then it generated the request body based on its understanding.

PYTHON
1convert_to_json(response_2['output'])
Output
1[{'taxonomies': ['SPORT']}]

Finally, the two paths to generation - deterministic and semantic - can be applied in parallel by the Agent, as shown below:

PYTHON
1query_3 = "Look for GLBRL0000GACHALE, nmg 0000234GLCZD0000GUREDTOAA and car products"
2
3response_3 = agent_executor.invoke(
4    {
5        "input": query_3,
6        "preamble": preamble,
7    }
8)
Output
> Entering new AgentExecutor chain...

I will use the regex_extractor tool to extract the codes from the user query. Then, I will create a JSON for each of the key-value pairs in the dictionary.
{'tool_name': 'regex_extractor', 'parameters': {'user_query': 'Look for GLBRL0000GACHALE, nmg 0000234GLCZD0000GUREDTOAA and car products'}}
{'nmgs': ['0000234GLCZD0000GUREDTOAA'], 'objref': ['GLBRL0000GACHALE']}Relevant Documents: 0
Cited Documents: 0
Answer: ```json JSON
[
    {
        "objref": ["GLBRL0000GACHALE"],
        "nmgs": ["0000234GLCZD0000GUREDTOAA"]
    },
    {
        "taxonomies": ["CARS"]
    }
]
```
Grounded answer: ```json JSON
  [
    {
        "objref": [<co: 0>"GLBRL0000GACHALE</co: 0>"],
        "nmgs": [<co: 0>"0000234GLCZD0000GUREDTOAA</co: 0>"]
    },
    {
        "taxonomies": ["CARS"]
    }
]
```
> Finished chain.
PYTHON
1convert_to_json(response_3['output'])
Output
1[{'objref': ['GLBRL0000GACHALE'], 'nmgs': ['0000234GLCZD0000GUREDTOAA']},
2  {'taxonomies': ['CARS']}]

Conclusions

In this notebook we showed how Agents can be used to solve a real-world use case, in which the goal is to create API requests based on the user’s query. We did it by providing the Agent with a deterministic tool to extract relevant alphanumeric strings in the query, and matching them to the right parameter name. In parallel, the Agent can leverage the semantic understanding of the query provided by the LLM powering it.