cancel
Showing results for 
Search instead for 
Did you mean: 

CSV Parsing Error, More Columns Than Data

alex_panganiban
Contributor

I am experiencing a parsing error that I would like to be able to gracefully handle before the data streams into the CSV Parser snap. We receive a file from a partner where the last row doesn't actually contain any data, but instead, it has a hidden character <0x1a>. See the attached file and screenprint. The result is that a "more columns than data" parsing error is generated.

alex_panganiban_0-1698356162585.png

We are removing BOM's prior to the CSV Parser. I was wondering if someone can suggest a similar technique for removing this <0x1a> character in the last row. I prefer to proactively code for this condition prior to the CSV Parser, so that in the case there are valid "more columns than data" in other files or on other rows, that they will not ignored by our exception logic.

alex_panganiban_1-1698356390941.png

 

Thanks, Alex

 

1 ACCEPTED SOLUTION

alex_panganiban
Contributor

Hey Y'all, thank. you so much for your responses. I made a discovery of something that I wasn't aware of before asking this question. When the CSV Parser detects an error, one of the fields that is outputted in the error view is $original.record. Now that I know about this field, I can leverage it to determine which errors need to be reported and which errors may be ignored. Initially I had wanted to handle my error condition before the CSV Parser, but now I can handle it just as effectively post CSV Parser.

Again, thanks so much for your support. I really love this facility as I know I can always count on the community to help me solve any challenge.

View solution in original post

3 REPLIES 3

bojanvelevski
Valued Contributor

Hey @alex_panganiban,

You can trim the record to fit the headers, which will result in additional record with the hidden character as a value for the first column, in this case "employee_id".

bojanvelevski_1-1698394066202.png

 

bojanvelevski_0-1698394047091.png

From here, use Filter snap to exclude the records that doesn't have a "first_name" for example:

!$.hasPath('first_name')

Let me know if this works for you.

Regards,

Bojan

 

koryknick
Employee
Employee

@alex_panganiban - you can also use the "When errors occur" property in the Views tab of the CSV Parser snap to ignore errors during parsing or even route bad records to an error view and handle them more gracefully.

koryknick_0-1698406451098.png

Hope this helps!

 

alex_panganiban
Contributor

Hey Y'all, thank. you so much for your responses. I made a discovery of something that I wasn't aware of before asking this question. When the CSV Parser detects an error, one of the fields that is outputted in the error view is $original.record. Now that I know about this field, I can leverage it to determine which errors need to be reported and which errors may be ignored. Initially I had wanted to handle my error condition before the CSV Parser, but now I can handle it just as effectively post CSV Parser.

Again, thanks so much for your support. I really love this facility as I know I can always count on the community to help me solve any challenge.