07-17-2023 04:10 AM
I’m designing ah pipeline that will handle multiple delimiters in a single CSV parser. I’m using a expression library that has the delimiter details of the respective files as below.
[
{
“src_file”: “aab”,
“Delimit”: “PIPE”,
“tgt_table”: “T_STG_AAB”
}
},
{
“src_file”: “abc”,
“Delimit”: “TAB”,
“tgt_table”: “T_STG_ABC”
}
},
{
“src_file”: “efg”,
“Delimit”: “COMMA”,
“tgt_table”: “T_STG_EFG”
}
}
]
On the mapper snap I can write expression to refer this expression library lib.sample.find(x => x.src_file==“incoming_filename”).get(‘Delimit’).
But when I try to use the same in CSV parser it doesnt accept.
can we really do this.
07-18-2023 10:41 PM
any update on the above use case please.
07-19-2023 12:11 AM
Hello @neelamanikandan_arumugam
You can try by introducing a child pipeline for processing the files, and you can pass the Delimiter from the parent as a pipeline parameter. Attached below are the sample pipelines.
–sl-comm-csv-delimiter-parent_2023_07_19.slp (4.9 KB)
–sl-comm-csv-delimiter-child_2023_07_19.slp (2.6 KB)
Let me know if this helps you.
Regards,
Aleksandar.