Forum Discussion

darshthakkar's avatar
darshthakkar
Valued Contributor
4 years ago
Solved

Generating individual files based on 'n' records from the input data (excel) file

Hi Community,

Looking forward to your suggestions (I’m brand new to snapLogic and wanting to learn more!)

→ Input is an excel file with multiple columns, I’m planning to fetch 2 columns only from those multiple columns (this has been achieved so far)

→ For instance, the total records in an excel file are 10, I would want 10 individual csv and html files (with those 2 columns only) generated with filename as (row1col1 value).csv, (row1col1 value).html, (row2col1 value).csv, (row2col1 value).html and so on and so forth.

Thank you in advance for your help 🙂

  • darshthakkar's avatar
    darshthakkar
    11 months ago

    The solution was achieved by using "Group By N" snap with the below settings:

    Some transformation happened after that and it was fed to a pipeline execute which generated individual html/csv files for me. This has worked seamlessly for 45k+ files in a single execution though it takes around 15m for pipeline completion.

    Don't forget to use the below snap "Document to Binary" before generating html files via the File writer:

    Thank you.

     

    Best Regards,

    Darsh

     



3 Replies

  • Aleksandar_A's avatar
    Aleksandar_A
    Contributor III
    4 years ago

    Hi @darshthakkar,

    Try using Pipeline Execute Snap, so you can write one file per one input document. Also you can pass snap.in.TotalCount as Parameter so you can name the files as you listed before.
    Below is the Pipeline Execute Snap Settings:

    The Child Pipeline that is used for writing the files should look like this:

    And the File Writer Snap Settings:

    Note * This is only for csv files, there is not a html formatter in the available Snaps. *
    For dynamic formatting a html file head over to https://community.snaplogic.com/t/how-to-generate-a-html-file-with-snaplogic/1186

    BR,
    Aleksandar

  • darshthakkar's avatar
    darshthakkar
    Valued Contributor
    11 months ago

    The solution was achieved by using "Group By N" snap with the below settings:

    Some transformation happened after that and it was fed to a pipeline execute which generated individual html/csv files for me. This has worked seamlessly for 45k+ files in a single execution though it takes around 15m for pipeline completion.

    Don't forget to use the below snap "Document to Binary" before generating html files via the File writer:

    Thank you.

     

    Best Regards,

    Darsh