I need to parse a CSV file that has header and trailer records throughout the file in unpredictable locations. What would be the best way to try and do this?
Unfortunately these records do not have the same number of data elements as the data records so the CSV parse pipline is failing.
<data records> 01018,2019/01/13,2019/01/13,6,1049,3243,2514,4554,52 01018,2019/01/13,2019/01/13,7,21.53,79.14,138.80,100.46,62.00 Store Trailer-PUW,01018,2019/01/15,49 Store HDR-PUW,1026,2019/01/15 01026,2019/01/13,2019/01/07,1,17,192,29,278,4 01026,2019/01/13,2019/01/07,2,1018,6153,5391,6777,141 <more data records>
Any suggestions? Thanks!