Forum Discussion

aprabhav's avatar
aprabhav
New Contributor III
4 years ago
Solved

Wait in the parent pipeline till all the documents are processed in the child pipelines

I need to load tables in postgres db. I receive the data as csv files (one each for each table). Some of the tables are master data tables and the other are transaction data tables. The transaction d...
  • aprabhav's avatar
    aprabhav
    4 years ago

    Thanks a lot, Pero. This helped and it worked. Thanks again!!!

pmancevski's avatar
pmancevski
New Contributor III
4 years ago

Hi @aprabhav,

There are couple of ways to achieve this and also I’m assuming that in the child pipeline you have some separate logic for transaction data and you can know which one has dependency. Fastest solution i can think of is

With the router you will separate the data between master data and transaction data, “Gate” snap will not complete executing until the execution of all upstream Snaps is done. After that in the mapper you can get only transaction data ($input1[0]), which you separate before in the router.

Thanks,
Pero M.

  • aprabhav's avatar
    aprabhav
    New Contributor III
    4 years ago

    Thanks a lot, Pero. This helped and it worked. Thanks again!!!