โ07-12-2022 06:33 AM
Hi Team,
Iโm looking to count records on a couple of scenarios listed below:
(1) Records fetched from a flat file (e.g. excel, csv) and writing the total counts of records into a new column
e.g. File Reader --> Mapper (transformation rules here with new column added to count the total number of records) --> Excel/CSV formatter --> File Writer
Iโve tried using snap.in.totalCount
and snap.outputViews
inside a mapper but didnโt get the expected results.
(2) Records fetched from source system like snowflake, salesforce, oracle, etc.
without using a count command in the query itself
Iโm thinking of using a Group By or an Aggregate snap to get the counts, would that be the right approach?
(3) Counting number of records processed after the operation has been completed. For instance, Iโm writing a flat file (excel/csv) but want a new column ingested into that file dynamically that states the total number of docs processed AND send an email to the team that states total number of docs processed.
e.g. File Reader/Salesforce Read --> Mapper --> excel/csv formatter --> File Writer --> Mapper (anticipating this should have some rules) --> Email Sender (sends count ONLY)
Thanking you in advance for your time and help on this one.
Best Regards,
Darsh
Solved! Go to Solution.
โ07-12-2022 02:11 PM
Hey @darshthakkar,
You have couple of options here:
Third one is a bit abstract and Iโm not a big fan of, but I listed it as an option anyway.
Hope this helps,
BR
โ07-12-2022 02:11 PM
Hey @darshthakkar,
You have couple of options here:
Third one is a bit abstract and Iโm not a big fan of, but I listed it as an option anyway.
Hope this helps,
BR
โ08-12-2022 10:16 AM
Thanks @bojanvelevski, all of them worked.
My personal favorite is suggestion 1 as 2 has an overhead to the performance of the pipeline.
Closing this thread now.