cancel
Showing results for 
Search instead for 
Did you mean: 

Dependency between pipelines

sanam
New Contributor

Hi All,

I am trying to add dependency between the pipelines using pipeline execute snap and adding a join to wait for parallel pipelines to finish then starting the next.

issue: problem with this solution being that the pipeline after the join keeps executing again and again for each document.
Can you please help me with the solution what is the best way to add dependency between pipelines or tasks.

10 REPLIES 10

tstack
Former Employee

I’m not quite clear on the configuration of your pipeline and how it’s behaving. Are the child pipelines before the Join generating more than one output document? How many documents is the Join outputting?

sanam
New Contributor

yes the first two pipelines are uploading data to tables and then the last one just takes a snapshot. I need everything to run once not document by document.
You can consider it that I want to create the flow of tasks as tables have dependency It should not be on the documents.
I am also trying task execute but not much of success.

tstack
Former Employee

Sorry, I’m still not clear on how your master pipeline is working. Are the child pipelines before the Join generating more than one output document? How many documents is the Join outputting? Maybe export the pipeline or take some screenshots and attach it so we can see the details.

You should be able to build the child pipelines so that they don’t have any unlinked outputs. In which case, the PipeExec will only output one document with the status of the child execution. So, as I mention, I’m not clear on why there are multiple documents flowing out of the Join.

sanam
New Contributor

Hi ,
Below are attachedmain%20pipline1 screenshot of my main pipline

Each pipeline is inserting data to tables and the count can vary.
Screenshot for child pipeline
child%20pipeline

below are the join configurations
join%20configration

Can you please suggest is pipeline execute the best way to achieve it and how to make sure that the pipeline runs only once after the join.

Regards,
Sanam