Advance Prompt Engineering
This guide will cover some advanced prompt engineering techniques and how to apply them in Snaplogic GenAI App Builder to help you tackle more complex tasks and enhance overall performance. You will learn how to use system prompts, structure responses in JSON, create complex prompts, manage tokens, and consider prompt and context size. First, let’s level set on what exactly prompt engineering is and why it’s important. What is Prompt Engineering? At its core, prompt engineering is about designing the input (the “prompt”) that you give to an AI model. The way you phrase your prompt can significantly impact the quality and relevance of the model’s output. It’s not just about what you ask the AI to do, but how you ask it. Why is Prompt Engineering Important? Even the most advanced AI models rely heavily on the prompts they receive. A well-crafted prompt can lead to insightful, accurate, and highly relevant responses, while a poorly structured prompt can result in vague, inaccurate, or irrelevant answers. Understanding the nuances of prompt engineering can help you maximize the effectiveness of your AI applications. Prerequisites Basic of Snaplogic OpenAI, Azure OpenAI, Amazon Bedrock Anthropic Claude, or Google Gemini account System prompt The System prompt is a special input that defines the LLM’'s behavior, tone, and boundaries before it engages with users. It establishes the context and sets the rules for interactions, ensuring that the assistant's responses align with the desired persona and goals. Imagine you’re a travel agent assistant. Your job is to provide clients with tailored and precise travel recommendations. To do this effectively, it’s essential to establish the LLM’s behavior through the System prompt: defining the assistant’s role, setting the appropriate tone and style, and including important instructions. Drag the "OpenAI Chat Completion" "Azure OpenAI Chat Completion" "Anthropic Claude on AWS Messages", or "Google Gemini Generate" onto the canvas Select "Account" tab and select your configured account Select “Settings” tab to configure these fields Select a model: click the chat bubble icon to view the list of available models and select your preferred one. Set “Prompt” “I’m organizing a 3-day honeymoon in Japan. We’re interested in cultural experiences, and nature.” Set “System Prompt”: expand the “Advanced prompt Configuration” and enter the System prompt “You are a travel agent assistant. Provide personalized travel recommendations based on user preferences. Maintain a friendly and conversational tone. If asked about travel plan, include accommodations and restaurants.” Save and close the snap settings panel Validate the pipeline and let’s see the result by clicking the preview data output To summarize the above example System prompt: You are a travel agent assistant. Provide personalized travel recommendations based on user preferences. Maintain a friendly and conversational tone. If asked about travel plan, include accommodations and restaurants. Question: I’m organizing a 3-day honeymoon in Japan. We’re interested in cultural experiences, and nature. Answer: That's so exciting! Japan is the perfect destination for a honeymoon filled with cultural experiences and beautiful scenery. For your 3-day trip, I recommend starting in Kyoto, known for its traditional temples and gardens. You can visit iconic sites like Fushimi Inari Shrine, Arashiyama Bamboo Grove, and Kinkaku-ji Temple. Be sure to catch a traditional tea ceremony or try on a kimono for a truly immersive experience. Next, head to Hakone for a taste of Japan's stunning natural beauty. You can explore the Hakone Open-Air Museum, take a relaxing dip in an onsen (hot spring), and enjoy a scenic cruise on Lake Ashi with views of Mt. Fuji. For accommodations, I suggest staying in a traditional ryokan for an authentic Japanese experience. Ryokans offer tatami-matted rooms, futon beds, and kaiseki dinners featuring seasonal dishes. As for dining, make sure to try local specialties like sushi, ramen, and tempura. And don't miss out on trying sake, Japan's famous rice wine! Let me know if you'd like more details or assistance with booking accommodations and restaurants. Have a wonderful honeymoon in Japan! 🎎🌸🗾 The response adheres to the system prompt by providing personalized recommendations in a friendly tone, including details on accommodations and dining options. JSON response Structuring response in JSON format is a crucial technique for enhancing clarity, maintaining consistency, and ensuring seamless integration with various systems, including front-end applications, APIs, and databases. For example, if you need to present information from the above travel agent assistant example in a predefined format on a website, generate a PDF, or connect with other services that facilitate booking hotels, and activities, it is essential to format the prompt response as JSON. This approach ensures compatibility and smooth interaction across different platforms and services. Let's try modifying the system prompt from the previous example to produce output in a specific JSON format. Click the Chat Completion snap to open settings. Update the system prompt to instruct the LLM to produce the JSON response: "You are a travel agent assistant. Provide a JSON response that includes destination, trip_duration, list of activities, list of hotels (with fields for name and description), and list of restaurants(with fields for name, location, and description)." Check the “JSON mode” checkbox. The snap will output a field named json_output that contains the parsed JSON object of response. Save and close the snap settings panel. Validate the pipeline and let’s see the result. The prompt answer is the JSON string and the parsed JSON object can be found in the “json_output” field since the JSON mode is enabled. The JSON response complies with the structure specified in the system prompt, ensuring that all necessary fields are included. The structured format supports seamless integration with downstream applications. For a travel agency, this capability allows for the efficient generation of personalized itineraries, which can be utilized to populate web pages, generate PDFs or Excel documents, send emails, or directly update travel booking systems, including querying flight availability and checking hotel options. Complex prompt Using a list of messages to incorporate conversation history helps maintain context in ongoing dialogues. This approach ensures responses are relevant and coherent, improving the overall flow of the conversation. Additionally, these messages can be provided as examples of user responses to guide the model in interacting effectively. By including previous interactions, it enhances continuity and user engagement, facilitating the model's ability to handle complex, multi-turn exchanges. This technique allows the model to generate more natural and accurate responses, especially when building on earlier details, resulting in a more seamless and intuitive conversation. Moreover, they can be used for example of response to let model know how should interact with user. Each message contain a role and content. The common roles are: System: Provides the initial context, setting the tone and behavior for the LLM. User: Represents the user’s input, guiding the conversation based on their queries or commands. Assistant/Model: Contains previous responses from the LLM or examples of desired behavior. This section will guide you through the process of constructing a message list and using it as input for the LLM. We'll create the following pipeline to make a travel agent assistant be able to answer questions by leveraging the context from previous conversations. In this example, user asks about Japan's attractions in April and later inquires about the weather without specifying a location or time. Let’s create the pipeline and see how it works. Drag the "JSON Generator" snap onto the canvas. Click on the "JSON Generator" to open it, then click on the "Edit JSON" button in the main Settings tab Highlight all the text from the template and delete it. Paste in this text. This prompt will be used as the user question. { "prompt": "Can you tell me what the weather’s going to be like?" } The "JSON Generator" should now look like this Click "OK" in the lower-right corner to save the prompt Save the settings and close the snap Drag the “OpenAI Prompt Generator” or “Azure OpenAI Prompt Generator” onto the canvas. Connect the Prompt Generator to the “JSON Generator” Click on the "Prompt Generator" to open settings. Change the label to “System Prompt” Click on the "Edit prompt" to open the prompt editor Highlight all the text from the template and delete it. Paste in this text. We will use it as the system prompt. You are a travel agent assistant. Provide personalized travel recommendations based on user preferences. The prompt editor should now look like this Click "OK" in the lower-right corner to save the prompt Select the “Advanced prompt output” checkbox. The “User role” field will be populated. Set the “User role” field to “SYSTEM” The final settings of the “System Prompt” should now look like this. Save the settings and close the snap Drag the second “Prompt Generator” onto the canvas and connect it to the prior snap. This snap will handle the previous user’s questions. Follow step 9 to 17 as a guide to configure the following fields Label: User Message 1 Prompt editor: I am planning a trip to Japan in April. Can you help me find some tourist attractions? User role: USER The final settings of the “User Message 1” should be like this. Drag the third “Prompt Generator” onto the canvas and connect it to the prior snap. This snap will handle the previous LLM’s answer. Follow step 9 to 17 as a guide to configure the following fields Label: Assistant Message Prompt editor: Sure! Some tourist attractions in Japan during your trip in April are: 1. Cherry Blossom Viewing 2. Fushimi Inari Shrine 3. Hiroshima Peace Memorial Park 4. Mount Fuji 5. Gion District Let me know if you need more information or assistance with planning your trip! User role: ASSISTANT The final settings of the “Assistant Message” should be like this. Drag the fourth “Prompt Generator” onto the canvas and connect it to the prior snap. This snap will handle the user question. Follow step 9 to 17 as a guide to configure the following fields: Label: User Message 2 Prompt editor: {{prompt}} User role: USER The final settings of the “User Message 2” should be like this. Drag the “Chat Completion” onto the canvas and connect it to “User Message 2”. Click on the "Chat Completion" to open settings. Select the account in the Account tab. Select the Settings tab. Select the model name. Check “Use message payload” checkbox. The prompt generator will create a list of messages in the "messages" field. Enabling "Use message payload" is necessary to use this list of messages as input. The “Message payload” field appears. Set the value to $messages. The settings of the Chat Completion should now look like this Save and close the setting panel Validate the pipeline and let’s see the result. Click on the output view of “User Message 2” to see the message payload, which we have constructed using the advanced mode of the prompt generator snap. Click on the output view of “Chat Completion” snap to see the LLM response. The result is: In April, the weather in Japan is generally mild and pleasant with cherry blossoms in full bloom. The temperatures are typically around 10-20°C (50-68°F) and there may be occasional rain showers. It's a great time to explore outdoor attractions and enjoy the beautiful spring scenery. Make sure to pack layers and an umbrella just in case! The model effectively delivered weather information for Japan in April, even the last user query did not specify location or time. This is possible because the model uses the entire conversation history to understand the context and flow of the dialogue. Furthermore, the model echoed the user’s question before responding, maintaining a consistent conversational style. To achieve the best results, make sure your message list is complete and well-organized, as this will help the LLM generate more relevant and coherent responses, enhancing the quality of the interaction. Tokens Tokens are units of text, including words, character sets, or combinations of words and punctuation, that language models use to process and generate language. They can range from single characters or punctuation marks to entire words or parts of words, depending on the model. For instance, the word "artificial" might be split into tokens like "art", "ifi", and "cial". The total number of tokens in a prompt affects the model's response capability. Each model has a maximum token limit, which includes both the input and output. For instance, GPT-3.5-Turbo has a limit of 4,096 tokens, while GPT-4 has limits of 8,192 tokens and 32,768 tokens for the 32k context version. Effective token management ensures responses remain within these limits, improving efficiency, reducing costs, and enhancing accuracy. To manage token usage effectively, the maximum tokens parameter is essential. It sets a limit on the number of tokens the model can generate, ensuring the combined total of input and output stays within the model’s capacity. Setting a maximum tokens parameter has several benefits: it prevents responses from becoming excessively long, reduces response times by generating more concise outputs, optimizes performance, and minimizes costs by controlling token usage. Additionally, it enhances user experience by providing clear, focused, and quicker responses. Use case Examples: Customer Support Chatbots: By setting maximum tokens, you ensure that the chatbot's responses are brief and focused, providing quick, relevant answers to user inquiries without overwhelming them with excessive detail. This enhances user experience and keeps interactions efficient. Content summarization: Helps generate concise summaries of long texts, suitable for applications with space constraints, such as mobile apps or notifications. Interactive Storytelling: Controls the length of narrative segments or dialogue options, maintaining engaging and well-paced storytelling. Product Descriptions: Generate brief and effective product descriptions for e-commerce platforms, maintaining relevance and fitting within space constraints. Let's walk through how to configure the maximum tokens in the SnapLogic Chat Completion snap using the prompt: “Describe the Photosynthesis in simple terms.”. We’ll see how the LLM behaves with and without the maximum token setting. Drag the “OpenAI Chat Completion” or “Azure OpenAI Chat Completion”, or “Google Gemini Generate” onto the canvas Select “Account” tab and select your configured account Select “Settings” tab Select your preferred model to use Set prompt to the message “Describe the Photosynthesis in simple terms.” The Chat Completion settings should now look like this Save the snap settings and validate the pipeline to see the result. In the result, the “usage” field provide us the token consumption detail. prompt_tokens: tokens used by the input completion_tokens: tokens used for generating response. total_tokens: the combined number of tokens used for both the input prompt and the generated response We can see that the response is quite long and the token used for response (completion_tokens) is 241. Let’s set the maximum token and see the result again Expand the “Model parameters” Set “Maximum tokens” to 100 Save the snap settings and validate the pipeline to see the result. The result is more concise compared to the output when the maximum tokens are not set. In this case, the number of completion_tokens used is only 84, indicating a shorter and more focused response. Using the maximum tokens effectively ensures that responses are concise and relevant, optimizing both performance and cost-efficiency. By setting this limit, you can prevent excessively long outputs, reduce response times, and maintain clarity in the generated content. To achieve optimal results, align the maximum tokens setting with your specific needs, such as the desired response length and application requirements. Regularly review and adjust this parameter to balance brevity with completeness, ensuring that the outputs remain useful and within operational constraints. Prompt size considerations In the previous section, we covered techniques for managing response size to stay within token limits. Now, we turn our focus to prompt size and context considerations. By ensuring that both prompts and context are appropriately sized, you can improve the accuracy and relevance of model responses while staying within token limits. Here are some techniques for managing prompt and context size: Keep prompt clear and concise By making prompts clear and direct, you reduce token usage, which helps keep the prompt within the model's limits. Focusing on essential information and removing unnecessary words enhances the accuracy and relevance of the model's responses. Additionally, specifying the desired output length further optimizes the interaction, preventing excessively long responses and improving overall efficiency. Example Prompt: “Could you please provide a detailed explanation of how the process of photosynthesis works in plants, including the roles of chlorophyll, sunlight, and water?” Better prompt: "Explain the process of photosynthesis in plants, including the roles of chlorophyll, sunlight, and water, in about 50 words." Splitting complex tasks into simpler prompts Breaking down complex tasks into smaller, more manageable subtasks not only reduces the size of each individual prompt but also enables the model to process each part more efficiently. This approach ensures that each prompt stays within token limits, resulting in clearer and more accurate responses. Example Complex Task: "Write a detailed report on the economic impact of climate change in developing countries, including statistical analysis, case studies, and policy recommendations." Simplified Prompts: "Summarize the economic impact of climate change in developing countries." "Provide a statistical analysis of how climate change affects agriculture in developing countries." "List case studies that demonstrate the economic consequences of climate change in developing countries." "Suggest policy recommendations for mitigating the economic impact of climate change in developing countries." Use a sliding window for chat history From the complex prompt section, we know that including the entire chat history helps maintain context, but it can also quickly use up available tokens. To optimize prompt size, employ a sliding window approach. This technique involves including only a portion of the chat history, focusing on recent and relevant exchanges, to keep the prompt within token limits. Summarize contexts Use a summarization technique to condense context into a brief summary. Instead of including extensive conversation history, create a concise summary that captures the essential information. This approach reduces token usage while retaining key details for generating accurate responses. By applying these techniques, you can effectively manage prompt and context size, ensuring that interactions remain efficient and relevant while optimizing token usage.
Introducing the Agent Snap
Flashback: What’s an Agent? “Agents are autonomous LLM-based processes that can interact with external systems to carry out a high-level goal.” Agents are LLM-based systems that can perform actions based on the user’s request and the scenario, determined by the LLM of the Agent system. A minimal agent consists of 1. an LLM component, and 2. tools that the Agent can use. Think of the Agent as a Robot with a brain (LLM) + robotic arms (Tools). Based on the request, the brain can “decide” to do something, and then the arm will carry out the action decided by the brain. Then, depending on the scenario, the brain can determine if more action is needed, or end if the request is complete. The process of an agent We previously introduced the “Agent Driver and Agent Worker“ pipeline pattern, which clearly defines every single operation that would occur in an Agent process. The process of the pattern can be described as follows Agent Driver Define the instruction of the Agent. (System prompt) Format the user’s request into a conversation. (Messages array) Define tools to make available to the Agent. Send all information above into a “loop“, run the Agent worker until the process is complete. Agent Worker Call the LLM with the instructions, conversation, and tool definitions LLM decides… If it is able to complete the request, end the conversation and go to step 7. If tool calls are required, go to step 3. Call the tools. Format the tool result. Add the tool results to the conversation Back to step 1. Request is complete, the agent responds. The rationale From the Agent Driver and the Agent Worker pipeline, here’s an observation: The driver pipeline handles all of the “configuration“ of the Agent. The worker pipeline handles the “operation“ of the Agent. Now, imagine this: If we can package the “Agent operation” into a single module, so that we can create Agents just by providing instructions, and tools. Wouldn’t this be great? This is exactly what Agent Snap does. The Agent Snap combines the PipeLoop Snap and the Agent Worker pipeline, so all of the agent operations happen in a single Snap. Information and prerequisites Now, before dreaming about having your own company of agents, since building agents is now so simple, there is some information to know about and conditions to be met before this can happen. 1. Agent Snaps are model-specific The Agent Snap is a combination of the “loop” and the Agent Worker, therefore, the LLM provider to be used for an Agent Snap is also fixed. This design allows users to stick to their favorite combination of customized model parameters. 2. Function(Tool) definitions must be linked to a pipeline to carry out the execution Previously, in an Agent Worker pipeline, the Tool Calling Snap is connected to Pipeline Execute Snaps to carry out tool calls, but this is no longer the case with the Agent Snap. Instead, a function definition should include the path of the pipeline to carry out the execution if this tool is called. This way, we can ensure every tool call can be performed successfully. If the user does not provide a tool pipeline with the function definition, the Agent Snap will not proceed. 3. Expected Input and Output of a tool pipeline When a tool call is requested by an LLM, the LLM will provide the name of the tool to call and the corresponding parameters to call. The Agent Snap will unwrap the parameters and send them directly to the tool pipeline. Here’s an example: I have a tool get_weather, which takes city: string as a parameter. The LLM decides to call the tool get_weather with the following payload: { "name": "get_weather", "parameters": { "city": "New York City" }, "sl_tool_metadata": { ... } } For this to work, my tool pipeline must be able to accept the input document : {"city": "New York City"} On a side note, the sl_tool_metadata object will also be available to the tool pipeline as the input for APIM and OpenAPI tools. Now, assume my tool pipeline has successfully retrieved the weather of New York City, It’s time for the Agent Snap to collect the result of this tool call. The Agent Snap will collect everything from the output document of the tool pipeline as the tool call result*. So that the LLM can determine the next steps properly. *Note: with one exception, if the output of a “tool pipeline“ contains the field “messages“ or "contents", it will be treated as the conversational history of the “child agent”, which will be filtered and will not be included. Build an Agent with Agent Snap We’ve understood the idea, we’ve gone through the prerequisites, and it’s time to build an Agent. In this example, we have an Agent with 2 tools: a weather tool and a calendar tool. We first start with a prompt generator to format the user input. Then define the tools the Agent can access. Let’s look into one of the tool definitions. In this example tool, we can see the name of the tool, the description of the tool, the parameters, and the path of the tool pipeline to carry out this task. This satisfies the requirement of a tool to be used by an Agent Snap. After we have the tools set, let’s look at the Agent Snap, using the Amazon Bedrock Converse API Agent Snap as an example. The configuration of an Agent Snap is similar to its corresponding Tool calling Snap, except for some extra fields, such as a button to visualize the agent flow, and a section to configure the operation of the Agent, such as iteration limit and number of threads for tool pipeline executions. The Agent Snap handles the whole executional process, and terminates when 1. The request is complete (no more tool calls are required) or 2. An error occurred. Voila! You have created an agent. After the Agent pipeline completes a round of execution, the user can use the “Visualize Agent Flow“ button in the Agent Snap to see the tools that are called by the LLM. Tips and Tricks for the Agent Snap Let’s take a look at the features built into the Agent Snap. Reuse pipelines Most agentic tool calls are processes that can be reused. To minimize execution load, we can use the “Reuse tool pipeline“ feature. This feature allows tool pipeline instances to be reused, so that the Agent will not need to spawn a pipeline every time a tool is called. To use this feature, the tool pipeline to be reused must be “Ultra compatible“; otherwise, the pipeline execution would hang, and the Agent Snap would eventually timeout. Tool call monitoring Agents can be long-running; it’s not rare to have an Agent run multiple iterations. To see what’s happening in the process, Agent Snap has built in monitoring during validation. The user will be able to see the iteration index, the tool that is currently being called, and the parameters that are used for the tool call in the pipeline statistics status bar. Selecting the “Monitor tool call“ option includes the parameter in the status update, which is an opt-in feature. If the user does not wish to expose the information to SnapLogic, the user should disable this. Warnings Agent configuration is a delicate process; a mistake can potentially lead to errors. The Agent Snap has a bunch of built-in warning capabilities, so the user can be better aware of what could go wrong. 1. Agent process completed before all tool calls completed In the Agent Snap, there is an Iteration limit setting, which limits the number of iterations the Agent can run. If the user provided a smaller limit, which caused the Agent to stop while the LLM is still awaiting tool calls, this warning would pop up to signal the user that the execution is incomplete. 2. Tool pipeline path is not defined A function (tool) definition to be used by the Agent Snap should include a tool pipeline path, so the Agent Snap can link to the actual pipeline that carries out the execution. If the pipeline path is not included in the function definition, this warning will pop up to signal the user that the Agent will not proceed. 3. Duplicate tool naming As we try to add more and more tools to the Agent Snap, two tools likely share the same name. The Agent Snap has the ability to rename the tools being sent to the LLM, and then still link to the correct pipeline. There will also be a warning available in the pipeline statistics to alert the user about a change in the behavior. Release Timeframes The Agent Snap is the foundation of the next-generation SnapLogic Agent. We will be releasing 4 Agent Snaps in the November release: Amazon Bedrock Converse API Agent OpenAI Chat Completions Agent Azure OpenAI Chat Completions Agent Google Gemini API Agent To better use the Agent Snaps, we will be introducing new capabilities to some of our Function Generators as well. Here is the list of Function Generator Snaps that will be modified soon: APIM Function Generator Snap OpenAPI Function Generator Snap MCP Function Generator Snap We hope you are as excited as we are about this one.
