Forum Discussion

mayur's avatar
mayur
New Contributor II
4 years ago

CSV parser Snap is not detecting duplicate fields

Sample CSV file:
“id”, “Stage”, “Stage”,“Type”
“1”,“close”,“config”, “test”
“1”,“open”,“plan”, “test”

We have duplicate columns name and different values… But CSV parser snap throws only one field.

How can i read both duplicate fields?

2 Replies

  • Hi @smeitei ,
    First you need to group all input documents into a single array. For that you can use the Group by N snap with the setting “Group size” seto to 0. Then use a mapper with the following expression: jsonPath($, "Records[*]").map(val => val.get("Record")).join("!")


    Output:

    You can also use the Aggregate snap, but this will use the default delimeter.

  • viktor_n's avatar
    viktor_n
    Contributor II

    Another one solution for this.
    jsonPath($, "$group[*].Record").reduce((acc, curr) => acc + "!" + curr)