This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
FAQ
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results forย 
Search instead forย 
Did you mean:ย 

Csv file with varying number of columns in each row

Go to solution
heidi_andrew
Contributor

โ€Ž02-06-2023 09:52 AM

my csv file has several rows for each person on the file. The specific rows have different number of columns in each row. CSV file reader is having trouble with it. Is this impossible to read for this snap?

0 Kudos
1 ACCEPTED SOLUTION

Go to solution
koryknick
Employee
Employee

โ€Ž02-22-2023 01:41 PM

@heidi.andrew - here is another pipeline that provides a couple more examples how this could be done. In this pipeline, there are two examples - the first is similar to what @alchemiz has done, but with an added step to simplify the syntax of the record conversion into an array then object. The second path steps it down even farther using all basic snap functionality to split the input stream so it can be parsed separately based on the input row type.

I hope this helps!
Community 15513 - Multi-row CSV_2023_02_22.slp (24.3 KB)

View solution in original post

17 REPLIES 17

Go to solution
darshthakkar
Valued Contributor

โ€Ž02-06-2023 02:00 PM

Hi @heidi.andrew,

I havenโ€™t seen this issue thus far with File reader snap. Can you please elaborate the snaps youโ€™re using for this?
Usually, I would go ahead with File Reader โ†’ CSV Parser โ†’ Mapper (with null-safe access checked on)
If youโ€™re using the same snaps, can you share the settings youโ€™re using for File Reader and CSV parser?

0 Kudos

Go to solution
heidi_andrew
Contributor

โ€Ž02-07-2023 04:07 AM

image

image
Here are a couple of people and their rows:
EEID|55555|xxxxxxxxx|MICHAEL|C|xxxxx|M|06/21/xxxx
OFFR|55555|01/01/2023-55|OE|01/01/2023|01/01/2023|01/01/2023|12/31/2023|02/06/2023 05:00:00.000000 AM
ELIG|55555|01/01/2023-55|CDHU02500L|BLUE BASIC|EEONLY|EMPLOYEE ONLY|Y|103.28|511.62|Y|Y|Y|Y|Y|||
COVG|55555|OE|01/01/2023|||||||01/01/2023|||||Y|06|UNKNOWN|02/06/2023 05:00:00.000000 AM|01/01/2023-55
EEID|7777777|xxxxxxxxx|KEITH|E|xxxxxxxx|M|11/11/xxxx
OFFR|7777777|xxxxxxxxx|KEITH|E|xxxxxxxx|M|11/11/xxxx
OFFR|7777777|01/01/2023-55|OE|01/01/2023|01/01/2023|01/01/2023|12/31/2023|02/06/2023 05:00:00.000000 AM
ELIG|7777777|01/01/2023-55|CDHU02500L|BLUE BASIC|EEONLY|EMPLOYEE ONLY|Y|203.28|411.62|Y|Y|Y|Y|Y|||
COVG|7777777|OE|01/01/2023|||||||01/01/2023|||||Y|06|UNKNOWN|02/06/2023 05:00:00.000000 AM|01/01/2023-55

0 Kudos

Go to solution
heidi_andrew
Contributor

โ€Ž02-07-2023 04:09 AM

I also played around with the ignore empty data and surrounding spaces features but that did not work for me either. there are no headings in the file but i gave it header names and thought that may help. i am joining them with another file. that does not give errors yet as i have been unable to get past that csv file reader.
image

0 Kudos

Go to solution
darshthakkar
Valued Contributor
In response to heidi_andrew

โ€Ž02-07-2023 06:33 AM

Hi @heidi.andrew,

Can you please share a sample .csv file if you donโ€™t mind. The snap properties look good to me however I might be overlooking something so wanted to try it on my end.

If itโ€™s a sensitive data, you can truncate and share me a reduced version of the file.
Thanks.

Regards,
Darsh

0 Kudos