cancel
Showing results for 
Search instead for 
Did you mean: 

Csv file with varying number of columns in each row

heidi_andrew
Contributor

my csv file has several rows for each person on the file. The specific rows have different number of columns in each row. CSV file reader is having trouble with it. Is this impossible to read for this snap?

1 ACCEPTED SOLUTION

koryknick
Employee
Employee

@heidi.andrew - here is another pipeline that provides a couple more examples how this could be done. In this pipeline, there are two examples - the first is similar to what @alchemiz has done, but with an added step to simplify the syntax of the record conversion into an array then object. The second path steps it down even farther using all basic snap functionality to split the input stream so it can be parsed separately based on the input row type.

I hope this helps!
Community 15513 - Multi-row CSV_2023_02_22.slp (24.3 KB)

View solution in original post

17 REPLIES 17

darshthakkar
Valued Contributor

@heidi.andrew - Apologies for the delay in getting back to you. I haven’t gotten a chance to look at the sample file you provided but seems like solution provided by @alchemiz worked. I will also test it on my end whenever I get a chance and keep you posted.

Thank you @alchemiz for your assistance.

Best Regards,
Darsh

koryknick
Employee
Employee

@heidi.andrew - here is another pipeline that provides a couple more examples how this could be done. In this pipeline, there are two examples - the first is similar to what @alchemiz has done, but with an added step to simplify the syntax of the record conversion into an array then object. The second path steps it down even farther using all basic snap functionality to split the input stream so it can be parsed separately based on the input row type.

I hope this helps!
Community 15513 - Multi-row CSV_2023_02_22.slp (24.3 KB)

i was finally able to get this to work. i used a fixed format parser that looked like this:
image
Now I have done the work on the rows and need to put them back together. As you can imagine, i have a ton of blank columns on all the rows now and dont want that. got any ideas?

heidi_andrew
Contributor

Thank you! I will check it out. i really needed a simpler pipeline to understand it and maybe yours will be just that.

I think this will work. Thanks!