Forum Discussion

Shawn_McCormick's avatar
Shawn_McCormick
New Contributor II
7 years ago

Parsing CSV file that has header/trailer records throughout

I need to parse a CSV file that has header and trailer records throughout the file in unpredictable locations. What would be the best way to try and do this?

Unfortunately these records do not have the same number of data elements as the data records so the CSV parse pipline is failing.

<data records>
01018,2019/01/13,2019/01/13,6,1049,3243,2514,4554,52
01018,2019/01/13,2019/01/13,7,21.53,79.14,138.80,100.46,62.00
Store Trailer-PUW,01018,2019/01/15,49
Store HDR-PUW,1026,2019/01/15
01026,2019/01/13,2019/01/07,1,17,192,29,278,4
01026,2019/01/13,2019/01/07,2,1018,6153,5391,6777,141
<more data records>

Any suggestions? Thanks!

3 Replies

  • Shawn_McCormick's avatar
    Shawn_McCormick
    New Contributor II
    7 years ago

    I am new to SnapLogic. I just found that I could route the error data to the error view. It’s not ideal but I can handle it that way.

    • Ajay_Chawda's avatar
      Ajay_Chawda
      Contributor
      7 years ago

      Hi Shawn.McCormick
      Possible to share pipeline or screen shot of pipeline how you achieve this requirement.

      • Shawn_McCormick's avatar
        Shawn_McCormick
        New Contributor II
        7 years ago

        For the CSV Parser pipline the default error behavior is to “stop pipeline execution”. But if you change it to “route error data to error view” it will kick out the records/documents that don’t fit the parser formatting and it will continue to process the good data.