How to loop thru N times
I have an input of 1000 Records coming in. I can only process 200 records at a time. How can I write a loop for this ?
Forum Discussion
My first go-to would be to create a sub pipeline for the part that needs to handle 200 records at a time and call that sub pipeline from the main pipeline using pipeline execute with the setting of batch size = 200.
Depending on what you need and your options, you may also be able to solve it by batching it in to groups of 200 using Group by N and processing each document as one batch.
I tried the Group By N. Batch size 2. But its not working.
