03-03-2022 09:54 AM
We are currently creating a Pipeline where we are using a Pipeline Execute. Pipeline Execute we have a batch number defined. In the pipeline that is triggered from PE we are writing a file. I would like to know if there is a way to get the batch number that is triggered i.e., lets say i have 650 records in source and batch number in PE is defined as 100. So for every 100 records it creates a file i want the file name created as filename_batch_1.csv so on and 7th file will have filename_batch_7.csv. Is it possible?
Solved! Go to Solution.
03-03-2022 07:34 PM
A couple of releases ago we added a Memory Sensitivity
property to the Group By N
snap. Set it to Dynamic
to dynamically reduce the group size according to available memory conditions.
03-14-2022 09:14 AM
Thank you. This worked well without impacting performance.
03-14-2022 01:45 PM
I’m glad to hear that! Thanks for the update.