New element in mapper

[ryagnik]

I have a pipeline in which I am reading a file and parse it with CSV parser.

Name|Age|Gender
David| 18| M
Julie|20|F

I am able to parse the file but at the end of each record, I would like to append a file name in which record comes.

So reading a file using file reader and copying a file name on copy snap to merge with join but somehow I only see a file name on first record.

I am trying a merge join but it merges output at first record, how do I merge file name on to mapped element ?

So expecting output something like this -

Name Age Gender Filename
David 18 M names.csv
Julie 20 F names.csv

Thanks…

[stodoroska]

You can also add Binary to document to the FileWriter, after that a copy snap, then document to binary and csv parser. After the csv parser you will add a join snap, where you will join the output of the copy snap and the output of the csv parser where you will have the $[‘content-location’] and the output of the parser.

But join is an expensive operation, so I am not sure with how many records are you dealing with in this pipeline. This is just a suggestion of a possible way how to implement it in a single pipeline.