Return the status of executed Pipelines for the specified time within the Org
I am new to snaplogic so please bear with me! I am trying to build a pipeline to extract the pipeline execution history for specified time range using GET REST snap. I could see below Query in the documentation: GET https://elastic.snaplogic.com/api/1/rest/public/runtime/orgname?start=&end=> has anyone used this api? how can I pass the value of start and end dates? I don’t want to hard code these dates as i want to make my pipeline more dynamic. I want to pass start date as first day of the previous month and end date as the last date of the previous month. I got above error while I try to create a snap. it says error on the index 87 which is “>” in the api query. So, I removed “>” and only used https://elastic.snaplogic.com/api/1/rest/public/runtime/orgname?start=&end= but still got below error: Failure: REST API service endpoint returned error result: status code = 404, Reason: REST API service endpoint returned error result: status code = 404, reason phrase = NOT FOUND, refer to the error_entity field in the error view document for more details, Resolution: Please check the values of Snap properties. Any help will be appreciated!
Pipeline monitoring to API Access Only user
I am trying to access pipeline monitoring information using the SnapLogic public API. I am currently using the /api/1/rest/public/runtime/ endpoint. When I use my own user access the endpoint, I get back all information for our organization’s running and completed pipelines. When I use a new user with API Access Only, I get no results back from the pipeline monitoring endpoint. I have also made sure that the new user is in the same group as my personal user. It still doesn’t work. Am I missing something to setting up a user for API Access Only?
Get metrics and resource utilization on current Snaplexes
Contributed by @pkona This pipeline pattern uses the APIs from SnapLogic metrics to query all Snaplexes and their current resource utilization. The data collected is saved into a database table. Data collected includes: Memory utilization Number of slots in use Node configurations like memory, CPU, disk space, Java version, etc. Configuration User configured on the REST account should be a SnapLogic admin. Specify the org name in the pipeline parameters. Source: SnapLogic API Target: SQL Server Snaps used: Rest Get, Mapper, JSON Splitter, Copy, Filter, Union, SQL Server Insert. Downloads Pattern - SL Org - Capture Snaplex runtime metrics.slp (38.9 KB)
SnapLogic Public APIs for Pipeline Runtime Execution Info
Created by @cstewart This pipeline uses the SnapLogic Public APIs to read the summary of executions, then does two things with them; first writes the line detail retrieved into a file, and concurrently, also writes a summary into a file, so you can see what the output contains. This pipeline utilizes the following techniques: Using the iteration feature of the REST Get Snap so that the multiple iterative requests required to retrieve the detail from the SnapLogic Public APIs are consumed. Dynamically configuring the URLs used with the expression language ternary operator Using an expression to get the org name (for use in the URLs) Putting multiple different format data into a single output stream, with an implication of order Configuration The REST Get Snap will need a REST Basic Auth account with credentials valid to make the request to the platform, and the result will only include the results for those executions which those credentials have authority to see. Sources: SnapLogic API Targets: CSV and JSON Files Snaps used: REST Get, JSON Splitter, Copy, Aggregate, Mapper, Union, JSON Formatter, CSV Formatter, File Writer Downloads GetRuntimeInfo.slp (19.8 KB)
Programmatically get SLPROPZ contents and write to a file
Within a pipeline, how can I programmatically access the SLPROPZ file for a snaplex, and write the contents of the file out to the local hard drive? We have over 500 snaplexes that we need to do this for, so manually downloading them all is not a good solution. Also, for various reasons I won’t go into, we cannot employ an outside task that just pings each download endpoint. This HAS to be done from within a set of pipelines. The SnapLogic Metadata snaps give me the ability to list Snaplexes and to read them as a set of JSON attributes. But there seems no way to obtain (or reconstruct) the SLPROPZ file. Any help that can be provided will be much appreciated!Prompt Deep Dive: Exporting audit logs
Overview Many SnapLogic customers are required by various industry regulations to retain audit logs for long periods of time. If you are a SnapLogic org administrator, you have either already built a pipeline to export your SnapLogic Activity Logs or are looking to build one. In one of our recent Office Hours sessions, a customer asked if SnapGPT could help create a pipeline to address this so we are going to take a few minutes to go through this example in a way that would produce a valid pipeline. Walkthrough Here is our SnapGPT prompt (screenshot below): “Create a pipeline that fetches my SnapLogic Activity Log and writes it to S3” SnapGPT comes back with a pipeline preview, which looks like a good starting point: After pressing “Import on new tab” we’re able to start the pipeline import process which includes having a chance to rename the pipeline and choose where you want to save it. Now we’ll open the REST Get snap to add our authentication and verify the URL. NOTE: If your REST Get snap does not include the URL, you can ask SnapGPT for the URL or copy it from here: https://{pod_path}/api/1/rest/public/activities/{org} . The placeholder {pod_path} is the beginning of the URL in your address bar for SnapLogic, so snapgpt.labs.snaplogic.com for SnapGPT in SnapLabs, or elastic.snaplogic.com for other environments. You may need to use the Elastic pod and a different org than SnapLabs if you want to validate and/or run this pipeline. I have used ‘elastic.snaplogic.com’ as my pod_path and ‘ConnectFasterInc’ for my {org} as seen in the screenshot below. If you do intent to run this from SnapLabs you will also want to check the “Trust all certificates” box. I’ve also set a query parameter ‘limit’ to a value of ‘500’. SnapGPT may add some additional expressions in the Mapper, so what you see below is a minimal change we can make to load raw entries and drop header and status information from the audit log file. Wrap up Your final step here would be to configure the S3 File Writer, or if you need to send the audit log to a different location you could reconfigure the Mapper and send to wherever you need the files to go. Video coming soon! Sometimes a video is worth a bunch of words and screenshots, so once we finalize the video we’ll post it here!
