Prompt Deep Dive: Exporting audit logs
Overview Many SnapLogic customers are required by various industry regulations to retain audit logs for long periods of time. If you are a SnapLogic org administrator, you have either already built a pipeline to export your SnapLogic Activity Logs or are looking to build one. In one of our recent Office Hours sessions, a customer asked if SnapGPT could help create a pipeline to address this so we are going to take a few minutes to go through this example in a way that would produce a valid pipeline. Walkthrough Here is our SnapGPT prompt (screenshot below): “Create a pipeline that fetches my SnapLogic Activity Log and writes it to S3” SnapGPT comes back with a pipeline preview, which looks like a good starting point: After pressing “Import on new tab” we’re able to start the pipeline import process which includes having a chance to rename the pipeline and choose where you want to save it. Now we’ll open the REST Get snap to add our authentication and verify the URL. NOTE: If your REST Get snap does not include the URL, you can ask SnapGPT for the URL or copy it from here: https://{pod_path}/api/1/rest/public/activities/{org} . The placeholder {pod_path} is the beginning of the URL in your address bar for SnapLogic, so snapgpt.labs.snaplogic.com for SnapGPT in SnapLabs, or elastic.snaplogic.com for other environments. You may need to use the Elastic pod and a different org than SnapLabs if you want to validate and/or run this pipeline. I have used ‘elastic.snaplogic.com’ as my pod_path and ‘ConnectFasterInc’ for my {org} as seen in the screenshot below. If you do intent to run this from SnapLabs you will also want to check the “Trust all certificates” box. I’ve also set a query parameter ‘limit’ to a value of ‘500’. SnapGPT may add some additional expressions in the Mapper, so what you see below is a minimal change we can make to load raw entries and drop header and status information from the audit log file. Wrap up Your final step here would be to configure the S3 File Writer, or if you need to send the audit log to a different location you could reconfigure the Mapper and send to wherever you need the files to go. Video coming soon! Sometimes a video is worth a bunch of words and screenshots, so once we finalize the video we’ll post it here!SnapLogic API Management FAQ
Here’s a list of technical FAQs we get from customers. Hopefully this helps you and if you can’t find the answer, feel free to respond to the post and I’ll try to answer! SnapLogic API Manager allows you to manage and secure all your APIs in a unified platform with Data Integration and iPaaS capabilities. Load Balancing and Distribution Yes, SnapLogic platform can scale horizontally and the APIM Gateway can be distributed across different server nodes/clusters. For our Cloudplex (Cloud API Gateway) we also can provision a load balancer to handle traffic distribution. What API security and governing policies do you have (as of March 2023): Anonymous Authenticator API Key Authenticator Authorize By Role Authorized Request Validator Callout Authenticator Client Throttling CORS Restriction Early Request Validator Generic OAuth2 IP Restriction Json Validator OAuth2 Client Credential Request Size Limit Request Transformer SQL Threat Detector XML DTD Validator XML XSD Validator API Testing: We currently offer real-time API testing through the try it out feature. Rate limiting: Yes, we do offer rate limiting and you can customize it to your needs. i.e. 250 anonymous requests per hour Observability: Yes, you can track requests, request errors, error(%), target errors, latency and Top API by requests. You can also export the API log data for further analysis on other data analytics platforms. Can you create APIs with a spec? Yes, you can either create, import from a json/yaml to create APIs. Or via a template OpenAPI Spec from within SnapLogic API Manager. Can you manage and secure 3rd-party APIs? Yes, you can manage and secure 3rd-party APIs and pipelines (APIs) you’ve created with SnapLogic API ManagerSnapLogic API Management Higher Education Case Study
Ever wonder how you can manage hundreds of APIs that enable +30,000 students on campus? Check out this case study with Boston University and their success with SnapLogic API management. SnapLogic Making the Faculty and Academia Even Smarter at Boston University Find out why Boston University selected SnapLogic to optimize the university’s application submission process for the Admissions and Financial Aid offices.
SnapLogic Public APIs for Pipeline Runtime Execution Info
Created by @cstewart This pipeline uses the SnapLogic Public APIs to read the summary of executions, then does two things with them; first writes the line detail retrieved into a file, and concurrently, also writes a summary into a file, so you can see what the output contains. This pipeline utilizes the following techniques: Using the iteration feature of the REST Get Snap so that the multiple iterative requests required to retrieve the detail from the SnapLogic Public APIs are consumed. Dynamically configuring the URLs used with the expression language ternary operator Using an expression to get the org name (for use in the URLs) Putting multiple different format data into a single output stream, with an implication of order Configuration The REST Get Snap will need a REST Basic Auth account with credentials valid to make the request to the platform, and the result will only include the results for those executions which those credentials have authority to see. Sources: SnapLogic API Targets: CSV and JSON Files Snaps used: REST Get, JSON Splitter, Copy, Aggregate, Mapper, Union, JSON Formatter, CSV Formatter, File Writer Downloads GetRuntimeInfo.slp (19.8 KB)
