Require API to build a pipeline to find the high CPU and memory utilization of an individual pipeline during its runtime
We are trying to design a pipeline in Snaplogic to identify the high CPU and memory utilization of an individual pipelines that ran in a month using public API. We tried using runtime API but we are not able to get the expected output. API used: “https://elastic.snaplogic.com/api/1/rest/public/runtime/ ” “https://elastic.snaplogic.com/api/1/rest/public/runtime/Environment/ ”+$Runtime_ID Is there an API for it? which can give the expected output.Prompt Deep Dive: Exporting audit logs
Overview Many SnapLogic customers are required by various industry regulations to retain audit logs for long periods of time. If you are a SnapLogic org administrator, you have either already built a pipeline to export your SnapLogic Activity Logs or are looking to build one. In one of our recent Office Hours sessions, a customer asked if SnapGPT could help create a pipeline to address this so we are going to take a few minutes to go through this example in a way that would produce a valid pipeline. Walkthrough Here is our SnapGPT prompt (screenshot below): “Create a pipeline that fetches my SnapLogic Activity Log and writes it to S3” SnapGPT comes back with a pipeline preview, which looks like a good starting point: After pressing “Import on new tab” we’re able to start the pipeline import process which includes having a chance to rename the pipeline and choose where you want to save it. Now we’ll open the REST Get snap to add our authentication and verify the URL. NOTE: If your REST Get snap does not include the URL, you can ask SnapGPT for the URL or copy it from here: https://{pod_path}/api/1/rest/public/activities/{org} . The placeholder {pod_path} is the beginning of the URL in your address bar for SnapLogic, so snapgpt.labs.snaplogic.com for SnapGPT in SnapLabs, or elastic.snaplogic.com for other environments. You may need to use the Elastic pod and a different org than SnapLabs if you want to validate and/or run this pipeline. I have used ‘elastic.snaplogic.com’ as my pod_path and ‘ConnectFasterInc’ for my {org} as seen in the screenshot below. If you do intent to run this from SnapLabs you will also want to check the “Trust all certificates” box. I’ve also set a query parameter ‘limit’ to a value of ‘500’. SnapGPT may add some additional expressions in the Mapper, so what you see below is a minimal change we can make to load raw entries and drop header and status information from the audit log file. Wrap up Your final step here would be to configure the S3 File Writer, or if you need to send the audit log to a different location you could reconfigure the Mapper and send to wherever you need the files to go. Video coming soon! Sometimes a video is worth a bunch of words and screenshots, so once we finalize the video we’ll post it here!
Return the status of executed Pipelines for the specified time within the Org
I am new to snaplogic so please bear with me! I am trying to build a pipeline to extract the pipeline execution history for specified time range using GET REST snap. I could see below Query in the documentation: GET https://elastic.snaplogic.com/api/1/rest/public/runtime/orgname?start=&end=> has anyone used this api? how can I pass the value of start and end dates? I don’t want to hard code these dates as i want to make my pipeline more dynamic. I want to pass start date as first day of the previous month and end date as the last date of the previous month. I got above error while I try to create a snap. it says error on the index 87 which is “>” in the api query. So, I removed “>” and only used https://elastic.snaplogic.com/api/1/rest/public/runtime/orgname?start=&end= but still got below error: Failure: REST API service endpoint returned error result: status code = 404, Reason: REST API service endpoint returned error result: status code = 404, reason phrase = NOT FOUND, refer to the error_entity field in the error view document for more details, Resolution: Please check the values of Snap properties. Any help will be appreciated!SnapLogic API Management FAQ
Here’s a list of technical FAQs we get from customers. Hopefully this helps you and if you can’t find the answer, feel free to respond to the post and I’ll try to answer! SnapLogic API Manager allows you to manage and secure all your APIs in a unified platform with Data Integration and iPaaS capabilities. Load Balancing and Distribution Yes, SnapLogic platform can scale horizontally and the APIM Gateway can be distributed across different server nodes/clusters. For our Cloudplex (Cloud API Gateway) we also can provision a load balancer to handle traffic distribution. What API security and governing policies do you have (as of March 2023): Anonymous Authenticator API Key Authenticator Authorize By Role Authorized Request Validator Callout Authenticator Client Throttling CORS Restriction Early Request Validator Generic OAuth2 IP Restriction Json Validator OAuth2 Client Credential Request Size Limit Request Transformer SQL Threat Detector XML DTD Validator XML XSD Validator API Testing: We currently offer real-time API testing through the try it out feature. Rate limiting: Yes, we do offer rate limiting and you can customize it to your needs. i.e. 250 anonymous requests per hour Observability: Yes, you can track requests, request errors, error(%), target errors, latency and Top API by requests. You can also export the API log data for further analysis on other data analytics platforms. Can you create APIs with a spec? Yes, you can either create, import from a json/yaml to create APIs. Or via a template OpenAPI Spec from within SnapLogic API Manager. Can you manage and secure 3rd-party APIs? Yes, you can manage and secure 3rd-party APIs and pipelines (APIs) you’ve created with SnapLogic API ManagerSnapLogic API Management Higher Education Case Study
Ever wonder how you can manage hundreds of APIs that enable +30,000 students on campus? Check out this case study with Boston University and their success with SnapLogic API management. SnapLogic Making the Faculty and Academia Even Smarter at Boston University Find out why Boston University selected SnapLogic to optimize the university’s application submission process for the Admissions and Financial Aid offices.
Programmatically get SLPROPZ contents and write to a file
Within a pipeline, how can I programmatically access the SLPROPZ file for a snaplex, and write the contents of the file out to the local hard drive? We have over 500 snaplexes that we need to do this for, so manually downloading them all is not a good solution. Also, for various reasons I won’t go into, we cannot employ an outside task that just pings each download endpoint. This HAS to be done from within a set of pipelines. The SnapLogic Metadata snaps give me the ability to list Snaplexes and to read them as a set of JSON attributes. But there seems no way to obtain (or reconstruct) the SLPROPZ file. Any help that can be provided will be much appreciated!Snaplogic API Proxy Use Case/Examples
Hi Everyone, I have a question relating API proxies in SnapLogic. I have now a lot of experience with creating APIs within the SnapLogic Platform and trying to create also Proxies. But was wondering if there are any existing use cases or examples of API proxies in SnapLogic. It’s just to expand my knowledge and to know when it’s better to create an API proxy or an API in SnapLogic. Thanks in advance! Regards JensPipeline monitoring to API Access Only user
I am trying to access pipeline monitoring information using the SnapLogic public API. I am currently using the /api/1/rest/public/runtime/ endpoint. When I use my own user access the endpoint, I get back all information for our organization’s running and completed pipelines. When I use a new user with API Access Only, I get no results back from the pipeline monitoring endpoint. I have also made sure that the new user is in the same group as my personal user. It still doesn’t work. Am I missing something to setting up a user for API Access Only?When I am trying to download a zip file , I am getting is not archive file invalid zip file when creating a zip file and writing it
I have created a pipeline and a trigger task (file writer) in the same project. But when I am trying to send the zip file using file writer / zip file writer snap in that api using REST PUT It is reflecting in manager and when I download it , it’s get downloaded successfully but while trying to open the file it is giving me the below error,I have pasted error while reading the zip file using zip file read but when i use file read using the same file it doesn’t give me any error: Is not archive Invalid zip file./The compressed zip file is invalid.
