cancel
Showing results for 
Search instead for 
Did you mean: 

PipeLine Execute- File Writter- Getting Batch Number

harshareddy
New Contributor II

We are currently creating a Pipeline where we are using a Pipeline Execute. Pipeline Execute we have a batch number defined. In the pipeline that is triggered from PE we are writing a file. I would like to know if there is a way to get the batch number that is triggered i.e., lets say i have 650 records in source and batch number in PE is defined as 100. So for every 100 records it creates a file i want the file name created as filename_batch_1.csv so on and 7th file will have filename_batch_7.csv. Is it possible?

1 ACCEPTED SOLUTION

A couple of releases ago we added a Memory Sensitivity property to the Group By N snap. Set it to Dynamic to dynamically reduce the group size according to available memory conditions.

View solution in original post

6 REPLIES 6

harshareddy
New Contributor II

Thank you. This worked well without impacting performance.

I’m glad to hear that! Thanks for the update.