Require API to build a pipeline to find the high CPU and memory utilization of an individual pipeline during its runtime
We are trying to design a pipeline in Snaplogic to identify the high CPU and memory utilization of an individual pipelines that ran in a month using public API. We tried using runtime API but we are not able to get the expected output. API used: “https://elastic.snaplogic.com/api/1/rest/public/runtime/ ” “https://elastic.snaplogic.com/api/1/rest/public/runtime/Environment/ ”+$Runtime_ID Is there an API for it? which can give the expected output.
Return the status of executed Pipelines for the specified time within the Org
I am new to snaplogic so please bear with me! I am trying to build a pipeline to extract the pipeline execution history for specified time range using GET REST snap. I could see below Query in the documentation: GET https://elastic.snaplogic.com/api/1/rest/public/runtime/orgname?start=&end=> has anyone used this api? how can I pass the value of start and end dates? I don’t want to hard code these dates as i want to make my pipeline more dynamic. I want to pass start date as first day of the previous month and end date as the last date of the previous month. I got above error while I try to create a snap. it says error on the index 87 which is “>” in the api query. So, I removed “>” and only used https://elastic.snaplogic.com/api/1/rest/public/runtime/orgname?start=&end= but still got below error: Failure: REST API service endpoint returned error result: status code = 404, Reason: REST API service endpoint returned error result: status code = 404, reason phrase = NOT FOUND, refer to the error_entity field in the error view document for more details, Resolution: Please check the values of Snap properties. Any help will be appreciated!
Programmatically get SLPROPZ contents and write to a file
Within a pipeline, how can I programmatically access the SLPROPZ file for a snaplex, and write the contents of the file out to the local hard drive? We have over 500 snaplexes that we need to do this for, so manually downloading them all is not a good solution. Also, for various reasons I won’t go into, we cannot employ an outside task that just pings each download endpoint. This HAS to be done from within a set of pipelines. The SnapLogic Metadata snaps give me the ability to list Snaplexes and to read them as a set of JSON attributes. But there seems no way to obtain (or reconstruct) the SLPROPZ file. Any help that can be provided will be much appreciated!Snaplogic API Proxy Use Case/Examples
Hi Everyone, I have a question relating API proxies in SnapLogic. I have now a lot of experience with creating APIs within the SnapLogic Platform and trying to create also Proxies. But was wondering if there are any existing use cases or examples of API proxies in SnapLogic. It’s just to expand my knowledge and to know when it’s better to create an API proxy or an API in SnapLogic. Thanks in advance! Regards JensPipeline monitoring to API Access Only user
I am trying to access pipeline monitoring information using the SnapLogic public API. I am currently using the /api/1/rest/public/runtime/ endpoint. When I use my own user access the endpoint, I get back all information for our organization’s running and completed pipelines. When I use a new user with API Access Only, I get no results back from the pipeline monitoring endpoint. I have also made sure that the new user is in the same group as my personal user. It still doesn’t work. Am I missing something to setting up a user for API Access Only?When I am trying to download a zip file , I am getting is not archive file invalid zip file when creating a zip file and writing it
I have created a pipeline and a trigger task (file writer) in the same project. But when I am trying to send the zip file using file writer / zip file writer snap in that api using REST PUT It is reflecting in manager and when I download it , it’s get downloaded successfully but while trying to open the file it is giving me the below error,I have pasted error while reading the zip file using zip file read but when i use file read using the same file it doesn’t give me any error: Is not archive Invalid zip file./The compressed zip file is invalid.
Blank pipe.ruuid returned from unbounded mapper
Hello All - In order to monitor our pipelines, I have placed a unbounded snap(mapper) at the start of the pipeline that “most of the time” returns the pipe.ruuid once the job kicks off. For some of our shorter jobs, I had to put a script at the start of the pipeline to act as a sleeper(10 seconds) so that we get the pipe.ruuid back before the pipeline ends so we can check the health of the pipeline. We are doing this because we are using a 3rd party scheduler to run the jobs and we have to be able to tell the scheduler if the job completed or failed or what not. The issue we are having is…randomly, the pipe.ruuid is returning a blank value. So when I go to run against the Public API to check the state of the pipeline, it’s getting passes a blank pipe.ruuid and that give me a 404 error. Is there a better way to retrieve the pipe.ruuid at the start of a job that will guarantee I won’t get a blank pipe.ruuid value, but instead the real pipe.ruuid value every time? Thanks all! Chris M
