Parsing CSV file that has header/trailer records throughout

[Shawn_McCormick]

I need to parse a CSV file that has header and trailer records throughout the file in unpredictable locations. What would be the best way to try and do this?

Unfortunately these records do not have the same number of data elements as the data records so the CSV parse pipline is failing.

<data records>
01018,2019/01/13,2019/01/13,6,1049,3243,2514,4554,52
01018,2019/01/13,2019/01/13,7,21.53,79.14,138.80,100.46,62.00
Store Trailer-PUW,01018,2019/01/15,49
Store HDR-PUW,1026,2019/01/15
01026,2019/01/13,2019/01/07,1,17,192,29,278,4
01026,2019/01/13,2019/01/07,2,1018,6153,5391,6777,141
<more data records>

Any suggestions? Thanks!

[Shawn_McCormick]

I am new to SnapLogic. I just found that I could route the error data to the error view. It’s not ideal but I can handle it that way.

[Ajay_Chawda]

Hi Shawn.McCormick
Possible to share pipeline or screen shot of pipeline how you achieve this requirement.

[Shawn_McCormick]

For the CSV Parser pipline the default error behavior is to “stop pipeline execution”. But if you change it to “route error data to error view” it will kick out the records/documents that don’t fit the parser formatting and it will continue to process the good data.