Forum Discussion

amardeep2021's avatar
amardeep2021
New Contributor III
4 years ago

Getting error in csv parser when connected to Decompress snap

Hi,

I am trying to read multiple zip files from S3 location using Directory browser-> S3 File reader → Decompress → csv Parser. Currently I have 2 zip files in the location which I could see till decompress preview. But csv parser is failing with below error. Does anyone know how to fix this and reason for the error? PFA for more info on settings and kindly help.

"Failure: Error parsing at record 5, content-location=s3:///bi-edw-dev/edw/inbound/services_demandplanner/zipdp/Upload1 210615-012.csv.zip, Reason: The number of CSV data fields is greater than the number of header columns in the input data: number of CSV data fields = 3, number of header columns = 1, Resolution: Please make sure the number of CSV data fields is not greater than the number of header columns."


Regards,
Amardeep Shanmugam.

2 Replies

  • vaidyarm's avatar
    vaidyarm
    Contributor
    5 years ago

    This did not work, used it for the below invalid id’s but do not work

    automatedclean@00d5g000005evyzea4
    autoproc@00d5g000005evyzea4
    noreply@00d5g000005evyzea4

    1. \w-. ↩︎

    • bojanvelevski's avatar
      bojanvelevski
      Valued Contributor
      5 years ago

      Here is the input:

      And the result:

      So I gave you the logic for matching a correct format of an email. I you want to stop the pipeline, or throw an error, than you have to make some adjustments.

      For example:

      $email.match(/[1]+@([\w-]+.)+[\w-]{2,4}$/g) == null ? “fail” : “pass” if you want to use expressions. Other way I can think of is a Data Validator. Select Pattern as a Constraint and enter the regex into Constraint value.

      1. \w-. ↩︎

      • vaidyarm's avatar
        vaidyarm
        Contributor
        5 years ago

        I tried it via the below test pipeline but it does not work, not sure what I am doing wrong

        Email Validator_2021_05_20.slp (6.5 KB)

        1. \w-. ↩︎