List started pipelines regardless of start date

[kmiesse]

I need to determine which pipelines are currently running regardless of when they started. It appears the Pipeline Monitoring API only returns pipelines that have started within the last_hours parameter (although I can’t seem to determine proper values for the start and end parameters - they are defined as “timestamp in milliseconds” - milliseconds since when?).
We have continuously running pipelines that may have been started more than 90 days ago, which I believe is beyond the start limit of the Pipeline Monitoring API. I need this information to determine if all pipelines are executing as they should. How can I determine which pipelines are running regardless of their start date? Thanks in advance!

[nganapathiraju]

IF you look at the documentation,

https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1438155/Pipeline+Monitoring+API

last_hours - integer - fetch data for last n hours. Max limit is 2160 hours (90 days).
default = 1
pipe_name - string - fetch data for a particular pipeline
org_wide - Boolean - fetch data for whole org including all phases
state - string - fetch data for particular pipeline states.
State: [NoUpdate, Prepared, Started, Queued, Stopped, Stopping, Completed, Failed]. (case sensitive)
    Queued - Pipeline is queued for processing, all pipelines start in this state
    NoUpdate - Intermediate state at startup
    Prepared - Pipeline has been prepared
    Started - Pipeline has started execution
    Completed - Pipeline completed successfully
    Failed - Pipeline failed
    Stopped - Pipeline was stopped, by user or by the system, indicated by the 'stopper' key
    Stopping - Pipeline is stopping
    Example: state=Completed,Failed

In your case, you would want to put a max time for last_hours and fetch your interested statuses

Hope that works out of you.

[kmiesse]

But that won’t work if a pipeline was started more than 90 days ago, which is the max, correct? It appears to only return pipelines started within the last_hours param. I need to list those that are currently running but may have started more than 90 days ago.

[nganapathiraju]

We dont keep anything beyond that time period which is why its available
through PUBLIC APIs.

If you wanted, you can store all that data through SPLUNK or some other
kind of monitoring tools.

Thanks & Regards,
Naveen

[aleung]

kmiesse:

tinuously running pipelines that may have been started more than 90 days ago, which I believe is beyond the start limit of the Pipeline Monitoring API. I need this information to determine if all pipelines are executing as they should. How can I de

We archive these data in our warehouse in a daily basis to allow analysis like this.