โ09-20-2022 10:13 AM
How would you create a separate file per record instead of the standard file with multiple records
โ09-20-2022 10:19 AM
@jason.steindorf: You can use โGroup by Nโ snap and use the below settings:
โ09-20-2022 11:00 AM
@jason.steindorf - what output format are you looking for? JSON Formatter has an option to output each document as a binary stream.
You could also use the binary header for some formatter snaps
Otherwise, you could probably accomplish this using Pipeline Execute with the โreuseโ option disabled so that each document is called by a new process
โ09-20-2022 12:32 PM
The output im looking for is csv.
I want to write 1 record per file.
The current pipeline is such as that all records are written to 1 file.
โ09-20-2022 01:04 PM
@jason.steindorf: As suggested earlier, you can use a โGroup by Nโ, and then a pipeline execute to write individual files in any format you want.
There would be a parent pipeline where you put โGroup by Nโ as the second last snap, Pipeline execute as the last snap. Your child pipeline will have a mapper --> csv formatter --> file writer
Please try using โGroup by Nโ and youโll see how those records get split into individual records.