Pipeline execution in series
Hi,
I have to execute two pipelines in series, currently using two pipeline execute snaps connected to each other, when I execute the pipeline these pipeline execute snap will execute both child pi...
Forum Discussion
If you mean you want all docs to go through the first before running the second, you could do a tail snap in between the pipeline executes to grab the last document out of the first before running the second. If you need the document output, so basically wanting to do batch as opposed to SnapLogic’s typical way of streaming data through, you could instead put a sort snap with unsorted input which will force the snap to read all documents first before sorting. Note that means all the documents will remain on the server for a time while they are gathered to sort, so would need to be sure you have the server resources to do so.
Thank you, The Tail snap worked for me.
I am working on loading data from CSV to Oracle DB, there is a performance issue when all pipelines are executed in parallel and also a scenario where the target table B is dependent on stage table A. So these leads for me to try in Series, like completion of TableA all docs and then Table B pipeline will start.
