Forum Discussion

darshthakkar's avatar
darshthakkar
Valued Contributor
4 years ago
Solved

Generating individual files based on 'n' records from the input data (excel) file

Hi Community, Looking forward to your suggestions (I’m brand new to snapLogic and wanting to learn more!) → Input is an excel file with multiple columns, I’m planning to fetch 2 columns only fro...
  • darshthakkar's avatar
    darshthakkar
    11 months ago

    The solution was achieved by using "Group By N" snap with the below settings:

    Some transformation happened after that and it was fed to a pipeline execute which generated individual html/csv files for me. This has worked seamlessly for 45k+ files in a single execution though it takes around 15m for pipeline completion.

    Don't forget to use the below snap "Document to Binary" before generating html files via the File writer:

    Thank you.

     

    Best Regards,

    Darsh

     



Aleksandar_A's avatar
Aleksandar_A
Contributor III
4 years ago

Hi @darshthakkar,

Try using Pipeline Execute Snap, so you can write one file per one input document. Also you can pass snap.in.TotalCount as Parameter so you can name the files as you listed before.
Below is the Pipeline Execute Snap Settings:

The Child Pipeline that is used for writing the files should look like this:

And the File Writer Snap Settings:

Note * This is only for csv files, there is not a html formatter in the available Snaps. *
For dynamic formatting a html file head over to https://community.snaplogic.com/t/how-to-generate-a-html-file-with-snaplogic/1186

BR,
Aleksandar

darshthakkar's avatar
darshthakkar
Valued Contributor
4 years ago

Thank you @AleksandarAngelevski, I will try with the solution you’ve provided.
Your help is much appreciated.