Overview
Many SnapLogic customers are required by various industry regulations to retain audit logs for long periods of time. If you are a SnapLogic org administrator, you have either already built a pipeline to export your SnapLogic Activity Logs or are looking to build one. In one of our recent Office Hours sessions, a customer asked if SnapGPT could help create a pipeline to address this so we are going to take a few minutes to go through this example in a way that would produce a valid pipeline.
Walkthrough
Here is our SnapGPT prompt (screenshot below): “Create a pipeline that fetches my SnapLogic Activity Log and writes it to S3”
SnapGPT comes back with a pipeline preview, which looks like a good starting point:
After pressing “Import on new tab” we’re able to start the pipeline import process which includes having a chance to rename the pipeline and choose where you want to save it.
Now we’ll open the REST Get snap to add our authentication and verify the URL.
NOTE: If your REST Get snap does not include the URL, you can ask SnapGPT for the URL or copy it from here:
https://{pod_path}/api/1/rest/public/activities/{org}. The placeholder{pod_path}is the beginning of the URL in your address bar for SnapLogic, sosnapgpt.labs.snaplogic.comfor SnapGPT in SnapLabs, orelastic.snaplogic.comfor other environments. You may need to use the Elastic pod and a different org than SnapLabs if you want to validate and/or run this pipeline.
I have used ‘elastic.snaplogic.com’ as my pod_path and ‘ConnectFasterInc’ for my {org} as seen in the screenshot below. If you do intent to run this from SnapLabs you will also want to check the “Trust all certificates” box. I’ve also set a query parameter ‘limit’ to a value of ‘500’.
SnapGPT may add some additional expressions in the Mapper, so what you see below is a minimal change we can make to load raw entries and drop header and status information from the audit log file.
Wrap up
Your final step here would be to configure the S3 File Writer, or if you need to send the audit log to a different location you could reconfigure the Mapper and send to wherever you need the files to go.
Video coming soon!
Sometimes a video is worth a bunch of words and screenshots, so once we finalize the video we’ll post it here!
