cancel
Showing results for 
Search instead for 
Did you mean: 

Csv file with varying number of columns in each row

heidi_andrew
Contributor

my csv file has several rows for each person on the file. The specific rows have different number of columns in each row. CSV file reader is having trouble with it. Is this impossible to read for this snap?

1 ACCEPTED SOLUTION

koryknick
Employee
Employee

@heidi.andrew - here is another pipeline that provides a couple more examples how this could be done. In this pipeline, there are two examples - the first is similar to what @alchemiz has done, but with an added step to simplify the syntax of the record conversion into an array then object. The second path steps it down even farther using all basic snap functionality to split the input stream so it can be parsed separately based on the input row type.

I hope this helps!
Community 15513 - Multi-row CSV_2023_02_22.slp (24.3 KB)

View solution in original post

17 REPLIES 17

koryknick
Employee
Employee

If you can attach a copy of the data in present and desired states, that would be very helpful.

There is a file attached to the ticket way early. That was the beginning state. I was able to pull that apart by rows, apply my fix and now all of the EEID rows are in one file, all of the OFFR rows are in one file, etc. Now i need to put them all back together like the one in the ticket.

koryknick
Employee
Employee

Can you provide more detail of what you’re doing now and what the current issue is, and perhaps why the solution provided earlier wasn’t implemented?

Also - since this topic already has a listed solution, you might want to open a new post with your current challenge. Make sure you provide the details of your input, the pipeline you’ve implemented so far, an example of the output you have, and the desired output.