โ11-26-2017 06:46 AM
Hi,
I am trying to create source target table row count validate pipeline for which I have used following design in Snaplogic:
Parent pipeine:
Child Pipeline:
1)Execute the query received from parent pipeline on the source tables and on the target tables.
2) Store the row count of source and target in one of the audit table in target.
My problem is In parent pipeline if the number of child pipeline execute is 100(in step 3) , the next snap(step 4) is executing 100 times due to which if my audit table has 10 non matching records, it is throwing 1000 records output which is wrong.
โ11-27-2017 01:14 PM
Hi Aditya,
You can use the โtailโ snap to limit document passthrough. It will only output 1 document, no matter how many input documents it receives. You might need this in multiple places. I think this will fix your problem.
If you need the 100 documents combined into 1 single document, you might want to check out the โaggregateโ snap.
โ11-28-2017 05:46 AM
Thanks, using tail snap it worked fine. I didnโt tried the aggregate or group by n, but i am sure using these snap will also work.
โ11-28-2017 12:58 AM
Take a look at using the โGroup By Nโ Snap at the end of your child pipeline, with a โGroup Sizeโ set to 0 (zero) it will combine all output documents into a single output document.
Hope this helps.