Forum Discussion

Matthias's avatar
Matthias
New Contributor III
4 years ago
Solved

File Reader + parser - assign filename

I’ve got a pipeline which is reading multiple files. A simplified version is the below: I’d need the output to contain a column with the filename of the original file. As the file content ...
  • RoyB's avatar
    RoyB
    3 years ago

    figured it out. the ID field needs to be assigned to $original.ID

    The "orginal", which is case sensitive, lifts it into a sub-document in the json

    which is then ignored by the SET, but you are still able to include it in the WHERE

    (I had tried this already, but had typed it as Original, which didn't work, but revisited it once I found sometihng in the help for the SQL SERVER UPDATE Snap

bojanvelevski's avatar
bojanvelevski
Valued Contributor
4 years ago
Copy Link

Hey @matthias.voppichler,

You can use Pipeline Execute. Add the CSV in a child pipeline, and pass the filename as a parameter, that way you can read the parameter and map it on every line that will come out of the CSV Parser using a Mapper. Files will be read one at a time, parsed, and the filename will be added to the lines accordingly.

You can even combine Marjan’s solution with pipeline execute and again you’ll have the same desired result.

Regards,
Bojan

  • Matthias's avatar
    Matthias
    New Contributor III
    4 years ago
    Copy Link

    This will work, yes - i was actually hoping to find a better solution without a child pipeline - but it appears like that’s the way to go.