I am building a pipeline where i am using Group by N. I would like to know if there is a way to get the batch number of each document.
For example lets say i have 100 records in source and my group size is 10. Now this 100 records are divided into 10 batches. I would like to get the number if each batch. Is there a way to do this?
Thanks in advance
You can achieve this by using a mapper after the group by, and adding the
snap.in.totalCount function. Of course don’t forget to map the group, or check the Pass through option.