Solved
split target csv file into more smaller CSV's
Hi Snaplogic experts, my pipeline is creating one big CSV file (50k rows) as the result. Collegues asked me, if it is possible to split this big CSV file into smaller ones (create csv files with th...
SL12345 - You can also use the Pipeline Execute snap and set the Batch Size value to your desired number of records in the target. Create a child pipeline that is just a CSV Formatter and File Writer snap and call that child in your Pipeline Execute.
Basically, this passes the number of records specified in the Batch Size property to an instance of the child pipeline which can create a file of those records and finish, then the parent will start a new child with the next "batch" of records, and will keep iterating until all input records are consumed. Simple data chunking of your original file.
Hope this helps!
