Introduction
In this article, we will be introducing the following.
-
Part 1: Four new classes of snaps for LLM function calling: Function Generator, Tool Calling, Function Result Generator, and Message Appender, which have been developed specifically for tool calling.
-
Part 2: The Function Calling pipeline to demonstrate how the new Function calling snaps work together to perform LLM function calling.
-
Part 3: Using PipeLoop snap to orchestrate Agent pipelines: iteratively call the Function Calling pipeline until the model generates a final result or meets other termination conditions to perform agentic workflows.
Part 1: Introducing 4 new snap classes for tool calling
-
Function Generator Snap: create a function definition.
-
Tool Calling Snap: sends tool calling request to the model to retrieve LLM model response.
-
Function Result Snap: formats the result of tool run to be sent back to the LLM.
-
Message Appender Snap: append the tool results to the messages payload array.
Function Generator Snap
The Function Generator Snap facilitates the creation of a Tool definition, enabling the model to understand and utilize the available tools.
Sample Output:
Spoiler
[
{
"tools": [
{
"sl_type": "function",
"name": "get_current_weather",
"description": "Get the curent weather in a given location",
"parameters": [
{
"name": "location",
"type": "STRING",
"description": "The city and state, e.g. San Francisco, CA",
"required": true
},
{
"name": "unit",
"type": "STRING",
"enum": [
"celcius",
"fahrenheit"
],
"required": false
}
]
}
]
}
]Tool Calling Snap
The Tool Calling Snap forwards user input and tool specifications to the model's API, receiving the model's generated output in return.
This snap has 2 output views:
-
The first view outputs
-
the full response from the model
-
the complete message payload, including the model's current response
-
-
The second view outputs
-
the list of tools to call
-
In the OpenAI and Azure OpenAI Tool Calling Snap, a JSON argument field is added by SnapLogic, whose value is a JSON object derived from converting the string-formatted argument of the model's response tool call.
-
-
Sample Input:
Spoiler
[
{
"tools": [
{
"sl_type": "function",
"name": "get_current_weather",
"description": "Get the curent weather in a given location",
"parameters": [
{
"name": "location",
"type": "STRING",
"description": "The city and state, e.g. San Francisco, CA",
"required": true
},
{
"name": "unit",
"type": "STRING",
"enum": [
"celcius",
"fahrenheit"
],
"required": false
}
]
}
],
"original": {
"messages": [
{
"content": "You are a helpful assistant.",
"sl_role": "USER"
}
],
"original": {
"prompt": "You are a helpful assistant.",
"original": {
"prompt": "What's the weather in San Francisco?"
}
}
}
}
]Sample Output - LLM Response View:
Spoiler
[
{
"output": {
"message": {
"role": "assistant",
"content": [
{
"text": "Okay, let's get the current weather for San Francisco."
},
{
"toolUse": {
"toolUseId": "tooluse_YOLmGccxRGWPmCKqxAKvgw",
"name": "get_current_weather",
"input": {
"location": "San Francisco, CA",
"unit": "fahrenheit"
}
}
}
]
}
},
"stopReason": "tool_use",
"usage": {
"inputTokens": 272,
"outputTokens": 90,
"totalTokens": 362
},
"metrics": {
"latencyMs": 1905
},
"messages": [
{
"role": "user",
"content": [
{
"text": "What's the weather in San Francisco?"
}
]
},
{
"role": "assistant",
"content": [
{
"text": "Okay, let's get the current weather for San Francisco."
},
{
"toolUse": {
"toolUseId": "tooluse_YOLmGccxRGWPmCKqxAKvgw",
"name": "get_current_weather",
"input": {
"location": "San Francisco, CA",
"unit": "fahrenheit"
}
}
}
]
}
],
"original": {
"tools": [
{
"sl_type": "function",
"name": "get_current_weather",
"description": "Get the curent weather in a given location",
"parameters": [
{
"name": "location",
"type": "STRING",
"description": "The city and state, e.g. San Francisco, CA",
"required": true
},
{
"name": "unit",
"type": "STRING",
"enum": [
"celcius",
"fahrenheit"
],
"required": false
}
]
}
],
"original": {
"messages": [
{
"content": "What's the weather in San Francisco?",
"sl_role": "USER"
}
],
"original": {
"prompt": "You are a helpful assistant.",
"original": {
"prompt": "What's the weather in San Francisco?"
}
}
}
}
}
]Sample Output - Tool Calls View:
Spoiler
[
{
"toolUse": {
"toolUseId": "tooluse_YOLmGccxRGWPmCKqxAKvgw",
"name": "get_current_weather",
"input": {
"location": "San Francisco, CA",
"unit": "fahrenheit"
}
}
}
]Function Result Generator Snap
The Function Result Generator Snap formats the results generated by user-invoked functions into a custom data output structure defined within SnapLogic.
Different models have different requirements for the data type of the Content field. For example, Bedrock Converse requires Content to be a stringor a json, OpenAI requires Content to be stringonly. The Snap will stringify the content from the user if the format in the field is not supported.
Sample Input:
Spoiler
[
{
"content": [
{
"location": {
"name": "San Francisco",
"region": "California",
"country": "United States of America",
"lat": 37.775,
"lon": -122.4183,
"tz_id": "America/Los_Angeles",
"localtime_epoch": 1728429316,
"localtime": "2024-10-08 16:15"
},
"current": {
"last_updated_epoch": 1728429300,
"last_updated": "2024-10-08 16:15",
"temp_c": 17.3,
"temp_f": 63.1,
"is_day": 1,
"condition": {
"text": "Sunny",
"icon": "//cdn.weatherapi.com/weather/64x64/day/113.png",
"code": 1000
},
"wind_mph": 9.8,
"wind_kph": 15.8,
"wind_degree": 261,
"wind_dir": "W",
"pressure_mb": 1012,
"pressure_in": 29.89,
"precip_mm": 0,
"precip_in": 0,
"humidity": 88,
"cloud": 0,
"feelslike_c": 17.3,
"feelslike_f": 63.1,
"windchill_c": 17,
"windchill_f": 62.6,
"heatindex_c": 17,
"heatindex_f": 62.6,
"dewpoint_c": 13.7,
"dewpoint_f": 56.7,
"vis_km": 13,
"vis_miles": 8,
"uv": 2.3,
"gust_mph": 17.9,
"gust_kph": 28.8
}
}
],
"original": {
"toolUse": {
"toolUseId": "tooluse_YOLmGccxRGWPmCKqxAKvgw",
"name": "get_current_weather",
"input": {
"location": "San Francisco, CA",
"unit": "fahrenheit"
}
}
}
}
]Sample Output:
Spoiler
[
{
"sl_role": "TOOL",
"function_id": "tooluse_YOLmGccxRGWPmCKqxAKvgw",
"is_error": false,
"content": {
"location": {
"name": "San Francisco",
"region": "California",
"country": "United States of America",
"lat": 37.775,
"lon": -122.4183,
"tz_id": "America/Los_Angeles",
"localtime_epoch": 1728429316,
"localtime": "2024-10-08 16:15"
},
"current": {
"last_updated_epoch": 1728429300,
"last_updated": "2024-10-08 16:15",
"temp_c": 17.3,
"temp_f": 63.1,
"is_day": 1,
"condition": {
"text": "Sunny",
"icon": "//cdn.weatherapi.com/weather/64x64/day/113.png",
"code": 1000
},
"wind_mph": 9.8,
"wind_kph": 15.8,
"wind_degree": 261,
"wind_dir": "W",
"pressure_mb": 1012,
"pressure_in": 29.89,
"precip_mm": 0,
"precip_in": 0,
"humidity": 88,
"cloud": 0,
"feelslike_c": 17.3,
"feelslike_f": 63.1,
"windchill_c": 17,
"windchill_f": 62.6,
"heatindex_c": 17,
"heatindex_f": 62.6,
"dewpoint_c": 13.7,
"dewpoint_f": 56.7,
"vis_km": 13,
"vis_miles": 8,
"uv": 2.3,
"gust_mph": 17.9,
"gust_kph": 28.8
}
}
}
]Message Appender Snap
The message appender snap adds the results of tool runs to the message list, serving as input for subsequent tool calls.
Sample Input - First Input View - Messages
Spoiler
[
{
"output": {
"message": {
"role": "assistant",
"content": [
{
"text": "Okay, let's get the current weather for San Francisco."
},
{
"toolUse": {
"toolUseId": "tooluse_YOLmGccxRGWPmCKqxAKvgw",
"name": "get_current_weather",
"input": {
"location": "San Francisco, CA",
"unit": "fahrenheit"
}
}
}
]
}
},
"stopReason": "tool_use",
"usage": {
"inputTokens": 272,
"outputTokens": 90,
"totalTokens": 362
},
"metrics": {
"latencyMs": 1905
},
"messages": [
{
"role": "user",
"content": [
{
"text": "What's the weather in San Francisco?"
}
]
},
{
"role": "assistant",
"content": [
{
"text": "Okay, let's get the current weather for San Francisco."
},
{
"toolUse": {
"toolUseId": "tooluse_YOLmGccxRGWPmCKqxAKvgw",
"name": "get_current_weather",
"input": {
"location": "San Francisco, CA",
"unit": "fahrenheit"
}
}
}
]
}
],
"original": {
"tools": [
{
"sl_type": "function",
"name": "get_current_weather",
"description": "Get the curent weather in a given location",
"parameters": [
{
"name": "location",
"type": "STRING",
"description": "The city and state, e.g. San Francisco, CA",
"required": true
},
{
"name": "unit",
"type": "STRING",
"enum": [
"celcius",
"fahrenheit"
],
"required": false
}
]
}
],
"original": {
"messages": [
{
"content": "What's the weather in San Francisco?",
"sl_role": "USER"
}
],
"original": {
"prompt": "You are a helpful assistant.",
"original": {
"prompt": "What's the weather in San Francisco?"
}
}
}
}
}
]Sample Input - Second Input View - Tool Result
Spoiler
[
{
"sl_role": "TOOL",
"function_id": "tooluse_YOLmGccxRGWPmCKqxAKvgw",
"is_error": false,
"content": {
"location": {
"name": "San Francisco",
"region": "California",
"country": "United States of America",
"lat": 37.775,
"lon": -122.4183,
"tz_id": "America/Los_Angeles",
"localtime_epoch": 1728429316,
"localtime": "2024-10-08 16:15"
},
"current": {
"last_updated_epoch": 1728429300,
"last_updated": "2024-10-08 16:15",
"temp_c": 17.3,
"temp_f": 63.1,
"is_day": 1,
"condition": {
"text": "Sunny",
"icon": "//cdn.weatherapi.com/weather/64x64/day/113.png",
"code": 1000
},
"wind_mph": 9.8,
"wind_kph": 15.8,
"wind_degree": 261,
"wind_dir": "W",
"pressure_mb": 1012,
"pressure_in": 29.89,
"precip_mm": 0,
"precip_in": 0,
"humidity": 88,
"cloud": 0,
"feelslike_c": 17.3,
"feelslike_f": 63.1,
"windchill_c": 17,
"windchill_f": 62.6,
"heatindex_c": 17,
"heatindex_f": 62.6,
"dewpoint_c": 13.7,
"dewpoint_f": 56.7,
"vis_km": 13,
"vis_miles": 8,
"uv": 2.3,
"gust_mph": 17.9,
"gust_kph": 28.8
}
}
}
]Sample Output
Spoiler
[
{
"output": {
"message": {
"role": "assistant",
"content": [
{
"text": "Okay, let's get the current weather for San Francisco."
},
{
"toolUse": {
"toolUseId": "tooluse_YOLmGccxRGWPmCKqxAKvgw",
"name": "get_current_weather",
"input": {
"location": "San Francisco, CA",
"unit": "fahrenheit"
}
}
}
]
}
},
"stopReason": "tool_use",
"usage": {
"inputTokens": 272,
"outputTokens": 90,
"totalTokens": 362
},
"metrics": {
"latencyMs": 1905
},
"messages": [
{
"role": "user",
"content": [
{
"text": "What's the weather in San Francisco?"
}
]
},
{
"role": "assistant",
"content": [
{
"text": "Okay, let's get the current weather for San Francisco."
},
{
"toolUse": {
"toolUseId": "tooluse_YOLmGccxRGWPmCKqxAKvgw",
"name": "get_current_weather",
"input": {
"location": "San Francisco, CA",
"unit": "fahrenheit"
}
}
}
]
},
{
"sl_role": "TOOL",
"function_id": "tooluse_YOLmGccxRGWPmCKqxAKvgw",
"is_error": false,
"content": {
"location": {
"name": "San Francisco",
"region": "California",
"country": "United States of America",
"lat": 37.775,
"lon": -122.4183,
"tz_id": "America/Los_Angeles",
"localtime_epoch": 1728429316,
"localtime": "2024-10-08 16:15"
},
"current": {
"last_updated_epoch": 1728429300,
"last_updated": "2024-10-08 16:15",
"temp_c": 17.3,
"temp_f": 63.1,
"is_day": 1,
"condition": {
"text": "Sunny",
"icon": "//cdn.weatherapi.com/weather/64x64/day/113.png",
"code": 1000
},
"wind_mph": 9.8,
"wind_kph": 15.8,
"wind_degree": 261,
"wind_dir": "W",
"pressure_mb": 1012,
"pressure_in": 29.89,
"precip_mm": 0,
"precip_in": 0,
"humidity": 88,
"cloud": 0,
"feelslike_c": 17.3,
"feelslike_f": 63.1,
"windchill_c": 17,
"windchill_f": 62.6,
"heatindex_c": 17,
"heatindex_f": 62.6,
"dewpoint_c": 13.7,
"dewpoint_f": 56.7,
"vis_km": 13,
"vis_miles": 8,
"uv": 2.3,
"gust_mph": 17.9,
"gust_kph": 28.8
}
}
}
],
"original": {
"tools": [
{
"sl_type": "function",
"name": "get_current_weather",
"description": "Get the curent weather in a given location",
"parameters": [
{
"name": "location",
"type": "STRING",
"description": "The city and state, e.g. San Francisco, CA",
"required": true
},
{
"name": "unit",
"type": "STRING",
"enum": [
"celcius",
"fahrenheit"
],
"required": false
}
]
}
],
"original": {
"messages": [
{
"content": "What's the weather in San Francisco?",
"sl_role": "USER"
}
],
"original": {
"prompt": "You are a helpful assistant.",
"original": {
"prompt": "What's the weather in San Francisco?"
}
}
}
}
}
]By leveraging the four new Snaps, we will be able to create pipelines that are capable of LLM function calling, which we will refer as Function Calling Pipelines.
Part 2: Function Calling Pipeline Example
This pipeline demonstrates how to use the new snaps to perform LLM function calling.
Setup
We will be using the following four snaps for LLM Function calling:
-
Function Generator Snap
-
Tool Calling Snap
-
Function Result Generator Snap
-
Message Appender Snap
The function calling pipeline incorporates two tools as pipelines:
-
get_current_weather (Using PipeExec)
-
This pipeline retrieves weather information for a given location.
-
Pipeline setup
-
An HTTP Client that connects to the weatherapi endpoint
-
A mapper that passes the JSON output to the content field
-
-
foo_tool
-
A toy tool that outputs “foo“ as the result, used to demonstrate multi-tool calling capabilities.
-
Pipeline setup
-
A mapper that outputs “foo” in the content output
-
Execution Flow
The execution flow of this pipeline follows the following steps:
-
The user provides the prompt (wrapped in a messages payload) in a JSON Generator Snap, creates tool definitions using the Function Generator Snap, which is then sent to the LLM through the Tool Calling Snap.
-
The Chat completions view of the Tool Calling Snap outputs the response from the LLM and adds the current response from the LLM into the messages payload, and is connected to the first input of the Message Appender Snap for processing, the Tool calls view is connected to a router to pass tool calls to the individual tools.
-
The tools are invoked, then results are formatted by the Function Result Generator Snap
-
The Message Appender Snap collects and appends all tool invocation results to the messages array from the Chat completions view output from the Tool Calling Snap and outputs the modified messages array.
The output of the Message Appender contains the message history from the User prompt, LLM model respones, and the tool calling output, which marks the end of this round of tool calling.
Part 3: Agent Pipelines
To orchestrate LLM function calling pipelines or LLM Agent Pipelines, we introduce two patterns as pipelines to enable this functionality.
Agent Driver Pipeline
The Agent Driver Pipeline Leverages PipeLoop Snap to allow iterative executions on a single pipeline. The prompt input is defined then sent to the Agent Worker Pipeline (a Function Calling pipeline). The output of the Function Calling pipeline is then collected and sent again as the second iteration input of the Function Calling pipeline, the iteration will continue until the stop condition set in PipeLoop is reached or when the iteration limit is reached.
Agent Worker Pipeline
The Agent Worker Pipeline is similar to a Function Calling pipeline, the only difference is that the message payload is sent from the Agent Driver Pipeline through PipeLoop Snap instead of a JSON Generator snap.
Agent Pipeline Example - get_weather
This example demonstrates a weather checking assistant. This agent is equipped with a single tool - get_weather, which retrieves the weather data of a given location.
Agent Driver Pipeline
In this example, the user will provide a payload like below, which is to ask about the weather of a given location. (Which is mocked using a JSON generator Snap)
{ "prompt": "What's the weather in San Francisco?" }
The system prompt for this weather assistant is then defined in the first Prompt Generator
"You are a helpful weather assistant that will answer questions about the weather of a given location. You will be assigned with a tool to check the weather of the location."
The user prompt for this case is simply the prompt payload from the user, which we will pass to the Agent Worker Pipeline through the PipeLoop Snap. We will stop the PipeLoop Execution when the finish reason of the LLM is stop or end_turn (depending on the LLM model)
Agent Worker Pipeline
In the Agent Worker Pipeline, the flow follows the following steps
First Iteration:
-
Create function definitions for the tools to be called. In this case, the get_weather function.
-
Pass the message payload (system and user prompts), and tools payload (function definitions) to the Tool Calling Snap. The Tool Calling Snap will then decide to either call a tool or generate a result. In the first case, it will return a tool call decision for the pipeline to process.
[ { "toolUse": { "toolUseId": "tooluse_YOLmGccxRGWPmCKqxAKvgw", "name": "get_current_weather", "input": { "location": "San Francisco, CA", "unit": "fahrenheit" } } } ] - The Worker pipeline will then diverge into two branches. The first branch will pass the messages payload of this round to the Message Appender Snap, and the second branch will pass the tool call request to the tool to invoke a call and get the weather of San Francisco.
-
The result of the tool call will be collected and formatted by the Function Result Generator Snap, then passed to the Message Appender Snap so that the the Tool Call result can be added into the Message Payload. For this round, the finish reason of the LLM is
tool_use, which means the execution should continue, and the output of the Message Appender will be sent directly to the input of the Agent Worker Pipeline.
Message Appender Output
Spoiler
[
{
"output": {
"message": {
"role": "assistant",
"content": [
{
"text": "Okay, let's get the current weather for San Francisco."
},
{
"toolUse": {
"toolUseId": "tooluse_YOLmGccxRGWPmCKqxAKvgw",
"name": "get_current_weather",
"input": {
"location": "San Francisco, CA",
"unit": "fahrenheit"
}
}
}
]
}
},
"stopReason": "tool_use",
"usage": {
"inputTokens": 272,
"outputTokens": 90,
"totalTokens": 362
},
"metrics": {
"latencyMs": 1905
},
"messages": [
{
"role": "user",
"content": [
{
"text": "What's the weather in San Francisco?"
}
]
},
{
"role": "assistant",
"content": [
{
"text": "Okay, let's get the current weather for San Francisco."
},
{
"toolUse": {
"toolUseId": "tooluse_YOLmGccxRGWPmCKqxAKvgw",
"name": "get_current_weather",
"input": {
"location": "San Francisco, CA",
"unit": "fahrenheit"
}
}
}
]
},
{
"sl_role": "TOOL",
"function_id": "tooluse_YOLmGccxRGWPmCKqxAKvgw",
"is_error": false,
"content": {
"location": {
"name": "San Francisco",
"region": "California",
"country": "United States of America",
"lat": 37.775,
"lon": -122.4183,
"tz_id": "America/Los_Angeles",
"localtime_epoch": 1728429316,
"localtime": "2024-10-08 16:15"
},
"current": {
"last_updated_epoch": 1728429300,
"last_updated": "2024-10-08 16:15",
"temp_c": 17.3,
"temp_f": 63.1,
"is_day": 1,
"condition": {
"text": "Sunny",
"icon": "//cdn.weatherapi.com/weather/64x64/day/113.png",
"code": 1000
},
"wind_mph": 9.8,
"wind_kph": 15.8,
"wind_degree": 261,
"wind_dir": "W",
"pressure_mb": 1012,
"pressure_in": 29.89,
"precip_mm": 0,
"precip_in": 0,
"humidity": 88,
"cloud": 0,
"feelslike_c": 17.3,
"feelslike_f": 63.1,
"windchill_c": 17,
"windchill_f": 62.6,
"heatindex_c": 17,
"heatindex_f": 62.6,
"dewpoint_c": 13.7,
"dewpoint_f": 56.7,
"vis_km": 13,
"vis_miles": 8,
"uv": 2.3,
"gust_mph": 17.9,
"gust_kph": 28.8
}
}
}
],
"original": {
"tools": [
{
"sl_type": "function",
"name": "get_current_weather",
"description": "Get the curent weather in a given location",
"parameters": [
{
"name": "location",
"type": "STRING",
"description": "The city and state, e.g. San Francisco, CA",
"required": true
},
{
"name": "unit",
"type": "STRING",
"enum": [
"celcius",
"fahrenheit"
],
"required": false
}
]
}
],
"original": {
"messages": [
{
"content": "What's the weather in San Francisco?",
"sl_role": "USER"
}
],
"original": {
"prompt": "You are a helpful assistant.",
"original": {
"prompt": "What's the weather in San Francisco?"
}
}
}
}
}
]Second Iteration:
-
The updated message payload is then sent again with the function definitions to the Tool Calling Snap, the Tool Calling Snap for this round will then generate a result since it has retrieved the weather of San Francisco. The Tool Call output of the Tool Calling Snap will be empty for this round since no tool calls are required for this iteration.
-
The message payload is sent directly to the Message Appender Snap, and the finish reason of the LLM is
end_turn, which means the LLM has successfully carried out the request. PipeLoop execution will stop and the result will be sent to the output of the PipeLoop Snap in the Agent Driver Pipeline.
And the execution is finished.
Summary
In this article, we have introduced the new Snaps for Tool calling - Function Generator, Tool Calling, Function Result Generator, and Message Appender. We have also talked about how to create tool-calling pipelines and Agent Pipeline patterns. Happy building!
