cancel
Showing results for 
Search instead for 
Did you mean: 

Dynamically change the Delimiter in CSV parser

neelamanikandan
New Contributor II

I’m designing ah pipeline that will handle multiple delimiters in a single CSV parser. I’m using a expression library that has the delimiter details of the respective files as below.
[
{
“src_file”: “aab”,
“Delimit”: “PIPE”,
“tgt_table”: “T_STG_AAB”
}
},
{
“src_file”: “abc”,
“Delimit”: “TAB”,
“tgt_table”: “T_STG_ABC”
}
},
{
“src_file”: “efg”,
“Delimit”: “COMMA”,
“tgt_table”: “T_STG_EFG”
}
}
]

On the mapper snap I can write expression to refer this expression library lib.sample.find(x => x.src_file==“incoming_filename”).get(‘Delimit’).

But when I try to use the same in CSV parser it doesnt accept.

image

can we really do this.

2 REPLIES 2

neelamanikandan
New Contributor II

any update on the above use case please.

Aleksandar_A
Contributor III

Hello @neelamanikandan_arumugam

You can try by introducing a child pipeline for processing the files, and you can pass the Delimiter from the parent as a pipeline parameter. Attached below are the sample pipelines.

–sl-comm-csv-delimiter-parent_2023_07_19.slp (4.9 KB)
–sl-comm-csv-delimiter-child_2023_07_19.slp (2.6 KB)

Let me know if this helps you.

Regards,
Aleksandar.