SnapLogic - Integration Nation

Matthias · ‎12-17-2021

I’ve got a pipeline which is reading multiple files.

A simplified version is the below:

I’d need the output to contain a column with the filename of the original file.
As the file content has nothing to do with the filename, i can’t use the normal “copy / join” pattern, as the join would have to be one without condition - which would however multiply my data per input file.

CSV Parser does not seem to allow me to pass through the filename - which would in reality be the simplest and best solution.
Has anyone built something similar - or has some ideas on how to accomplish this?

bojanvelevski · ‎12-17-2021

Hey @matthias.voppichler,

You can use Pipeline Execute. Add the CSV in a child pipeline, and pass the filename as a parameter, that way you can read the parameter and map it on every line that will come out of the CSV Parser using a Mapper. Files will be read one at a time, parsed, and the filename will be added to the lines accordingly.

You can even combine Marjan’s solution with pipeline execute and again you’ll have the same desired result.

Regards,
Bojan

View solution in original post

Matthias · ‎12-17-2021

This will work, yes - i was actually hoping to find a better solution without a child pipeline - but it appears like that’s the way to go.

SnapLogic - Integration Nation

File Reader + parser - assign filename