On the Pipeline Execute snap, if I have the Reuse checkbox unchecked and the pool size =1, does that ensure the executions of the child pipelines are serial and not parallel? I want to ensure only one instance of the child pipeline is running at any time and that one execution completes before another begins.

Yes, a pool size of 1 means that only one child will be executed at a time and the Pipeline Execute snap will wait for the current child execution to finish before starting the next one.

vyasyash: But the drawback is you can’t leverage the parameters dynamically in this particular scenario. Can you elaborate? The OP mentioned that reuse was off, so the parameters passed to the child pipeline can change for each execution.

But the drawback is you can’t leverage the parameters dynamically in this particular scenario.

Actually we’ve developed a reusable component - service to reprocess failed transaction and then individual interface pipelines. In case of failures with interface pipelines, we will use the re-processing service to resend the failed transactions. By using pipeline execute in the reprocess service we want to achieve parallel execution on different failed pipelines with each failed transaction calling individual interface pipeline sequentially. We get parallel execution of each pipeline and sequential processing of documents in the order of input (with reuse option set to 1), but this does not support change in pipeline parameter values. Hence we will have to send the payload, runid and parameters as input documents instead of pipeline parameters. We’ve raised a ticket 20085 for this issue. If you’ve a solution for our situation please share will be really helpful.

The documentation is clear in this aspect. If it is not, please mention what to improve upon. If Reuse is enabled, pipeline parameter values cannot change between execution instances. If Reuse is disabled, you can pass different parameter values. If you want the parameters to be changed with reuse checked, then that is something an enhancement which may not be possible just by the design of this; which Product Management can elaborate. Hope that makes sense.

Ensure serial child pipeline execution | SnapLogic - Integration Nation

11 Replies

tstack
Former Employee
8 years ago
Yes, a pool size of 1 means that only one child will be executed at a time and the Pipeline Execute snap will wait for the current child execution to finish before starting the next one.
- vyasyash
  New Contributor III
  8 years ago
  But the drawback is you can’t leverage the parameters dynamically in this particular scenario.
  - tstack
    Former Employee
    8 years ago
    
    vyasyash:
    
    But the drawback is you can’t leverage the parameters dynamically in this particular scenario.
    
    Can you elaborate? The OP mentioned that reuse was off, so the parameters passed to the child pipeline can change for each execution.
kzaiger
New Contributor
8 years ago
What is the best way to ensure order of processing, e.g. pipeline a always starts and ends before pipeline b starts?
- nganapathiraju
  Former Employee
  8 years ago
  Place them in pipeline executes one after the other.

Forum Discussion

Ensure serial child pipeline execution

11 Replies

Recent Discussions

Google Sheets Subscribe questions

Basic string transformations not working

Can we generate XML file in pretty print format using native snapLogic snaps?

Multipart Reader failure - 'content-type' was not found

Unable to write sldb file to file system