List started pipelines regardless of start date

[kmiesse]

I need to determine which pipelines are currently running regardless of when they started. It appears the Pipeline Monitoring API only returns pipelines that have started within the last_hours parameter (although I can’t seem to determine proper values for the start and end parameters - they are defined as “timestamp in milliseconds” - milliseconds since when?).
We have continuously running pipelines that may have been started more than 90 days ago, which I believe is beyond the start limit of the Pipeline Monitoring API. I need this information to determine if all pipelines are executing as they should. How can I determine which pipelines are running regardless of their start date? Thanks in advance!

[kmiesse]

I added the plex_label param and removed the last_hours param, assuming it still wouldn’t return executions starting prior to the 1 hour default of last_hours and that are still running. And they were not returned, as expected, but other executions that were started in the last hour were returned. I don’t think this API will simply list running pipelines regardless of start time. Is there another option?

[kmiesse]

I added Enhancement Request: List Started Pipelines. I also submitted a Support ticket for it.

[ken · Accepted solution]

As an interim solution, have you tried manipulating the data returned by the results of the Snaplex Monitoring API? There is quite a bit of information returned, but at a Snaplex node level, you will have visibility of all active runtime IDs in the active_pipelines array. The runtime IDs could then be resolved using the Pipeline Monitoring API to mine additional, granular details.

[kmiesse]

Wow! Thank you! That just might do the trick.

[kmiesse]

@ken, do you know if the active_pipelines contains IDs regardless of their start time so I should see IDs from executions starting before the previous 90 days that the Pipeline Monitoring API is restricted to?