Forum Discussion

rvazzana's avatar
rvazzana
Employee
8 years ago

Pipeline queuing and Pipeline Exeute

Hello team,

We have documentation regarding queueing of pipelines here:
https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1439392/Pipeline+Queueing

It notes:
The SnapLogic control plane and data plane were improved in the Winter 2016/4.4 release to more gracefully handle certain overload conditions on a Snaplex. This change introduces a new “Queued” state for pipeline executions and adds properties to the Snaplex configuration to set the resource thresholds used to detect an overload.
When an execution is in the Queued state, the control plane will try to start the execution when resources become available on the Snaplex.

A pipeline will not stay in the Queued state forever; it will timeout after a while. A scheduled pipeline will timeout before the next scheduled execution or 4 hours, whichever comes first. A ForEach execution expires after 5 minutes.

Not all methods of execution will result in a pipeline being put into the Queued state. Only executions started by a Scheduled Task or a ForEach Snap.

My customer would like to know regarding the last sentence, what about executions that are started by a Pipeline Eecute - will that method of execution result in a pipeline being put into a queued state? I believe so, but I wanted to verify.

Thanks
Rob

Internal Q&A

3 Replies

  • dmiller's avatar
    dmiller
    Former Employee
    8 years ago

    @tstack I believe this is you, correct?

    (This category is private to SnapLogic employees.)

  • tstack's avatar
    tstack
    Former Employee
    8 years ago

    Not really. It can only be put in the Queued state if the child pipeline has no unlinked input or output views, which is fairly rare. Instead, PipeExec will retry the execution for up to an hour if the Snaplex is overloaded. So, you won’t see the Queued executions in the dashboard, but you will see a status message in the parent runtime dialog for the PipeExec snap (see the image in the Description section of the PipeExec docs).

    • rvazzana's avatar
      rvazzana
      Employee
      8 years ago

      Thank you for the information, Tim!!!

      Rob