Forum Discussion

GanguP's avatar
GanguP
New Contributor III
2 years ago
Solved

Parse complex file format and generate sequence number

I am trying to parse a file with a complex format.  It hasn't been easy as I am new to Snaplogic.  In the below example, I am trying to combine the four rows.  In order to do that I am thinking to assign the same seqeunce number to the group highlighted in yellow then use Aggregate to combine based on the sequence number.   How do I generate the sequence number?

 

 

  • j_angelevski's avatar
    j_angelevski
    3 years ago

    @ykoppisetty

    It is because the parse method expects a number but you are providing a string value.
    You need to parse the “Modified_Date”, you can do so by using parseInt(string).

    In your case would be:

    parseInt($Modified_Date)

    And the full expression to format the date would be:

    Date.parse(parseInt($Modified_Date))

10 Replies

  • koryknick's avatar
    koryknick
    Employee
    2 years ago
    Copy LinkReport

    GanguP - SnapLogic is a streaming platform, meaning it only retains context of the current document being processed.  I'd like to fully understand the requirement before showing you one solution vs another.

    Before offering a solution, can you clarify your use case?  Do you need to always combine 4 rows following a blank linedata value?  Or could there be any number of rows (0-N)?

    • GanguP's avatar
      GanguP
      New Contributor III
      2 years ago
      Copy LinkReport

      Hi Kory.  Thank you for a quick response.  Below is the format of the file I am trying to parse.   Below are the steps I am taking
      1) Use Fixed width parser to filter out lines that are not needed.

      2) Used mapper to assign the LOCATOR2_FLAG.  Below is the logic. 
      ($LINEDATA.substring(0,9).trim()>='0000000000' && $LINEDATA.substring(0,9).trim()<='9999999999') &&$LINEDATA.substr(38,1) .trim() == "-"?'Y':'N'

       

       

       

  • koryknick's avatar
    koryknick
    Employee
    2 years ago
    Copy LinkReport

    GanguP - could you please attach the file to this thread?  Or at least a portion of it?  I'd like to create a solution for you but would rather not hand-enter all the data from your sample.

  • koryknick's avatar
    koryknick
    Employee
    2 years ago
    Copy LinkReport

    Like any other development platform, there are many solutions; here is mine for this use case.  Please download and decompress the attached zip file, then import the SLP file as a new pipeline.  Update the File Reader with your filename and validate to ensure it is working properly.  You will need to finish adding the fields to the last Fixed Width Formatter, but I was able to see that the first few are coming out properly.

    The first Fixed Width Parser is skipping the first two lines to get rid of that initial header, then using the Ignore Lines table to skip over the other headers that repeat:

    Now the returned records should only contain the data we need: 4 lines per "group".  So we simply use the Group By N snap with a group size of 4 to push every 4 records into one document as an array.

    From here, we use a Mapper snap to concatenate the 4 records into a single string.  Note the use of the Array.reduce() method in the expression:

    $group.reduce((accum,cur)=> accum.concat(cur.record), '')

    Basically, this just starts with an empty string, and performs the String.concat() for each "record" element in the array.

    Note that I've also configured the Target Path in the Mapper as $content and set the output view of the Mapper to "Binary", which allows me to send the records into the last Fixed Width Parser snap so it can be easily split into the actual data elements that you need for the rest of your processing.

    Hope this helps!