09-06-2018 05:02 PM
I have a pipeline in which I am reading a file and parse it with CSV parser.
Name|Age|Gender
David| 18| M
Julie|20|F
I am able to parse the file but at the end of each record, I would like to append a file name in which record comes.
So reading a file using file reader and copying a file name on copy snap to merge with join but somehow I only see a file name on first record.
I am trying a merge join but it merges output at first record, how do I merge file name on to mapped element ?
So expecting output something like this -
Name Age Gender Filename
David 18 M names.csv
Julie 20 F names.csv
Thanks…
09-11-2018 01:43 AM
You can also add Binary to document to the FileWriter, after that a copy snap, then document to binary and csv parser. After the csv parser you will add a join snap, where you will join the output of the copy snap and the output of the csv parser where you will have the $[‘content-location’] and the output of the parser.
But join is an expensive operation, so I am not sure with how many records are you dealing with in this pipeline. This is just a suggestion of a possible way how to implement it in a single pipeline.