Generate multiple files based on single column

Question

I need to generate multiple files based on category, say for example, The table having list of columns and category column have multiple values.
Category:

milk_products
water_products
oil_products
rice_products

So from above category, I need to generate 4 files with its records in separate files dynamically in Snap Logic.
Can any one have any idea/suggestion how to achieve this?

tstack · Answer

Surendar:

I need to generate multiple files based on category, say for example, The table having list of columns and category column have multiple values.

I’m attaching an example pipeline that generates a separate JSON file for each document in a particular category:
FilePerCategory_2017_11_17.slp (7.0 KB)
Here’s some detail on what’s going on in the pipeline:

The Group By Fields Snap can be used to do the category grouping (make sure the input documents are sorted by category, otherwise I think you’ll get multiple outputs for the same category).
Once the categories are grouped together in a single document, you can write them out to a file using the JSON Formatter.  This JSON Formatter has the ‘Format each document’ option checked, which means that a separate binary output document will be generated for each input document.  We want this since every input document will be the collection of items in the category and that’s what we want in each file.  In addition, we set the ‘Content’ property to ‘$group’ since that contains the array of items in the category.  The last thing for the formatter is to compute the file name based on the category using the ‘Binary Header Properties’ to set the ‘content-location’ property in the binary output document header.
Finally, there’s a File Writer that writes out a file using the name as found in the ‘content-location’ property.

nganapathiraju · Answer

can you post the sample structure? There can be many ways of doing it!

tstack · Answer

Surendar:

I need to generate multiple files

What is the format of these files (e.g. CSV, JSON, XML, …)?

surendar · Answer

Any files either CSV or JSON files. by default is CSV

surendar · Answer

Thank you for the input and solution for json file. Is there any approach for CSV file format?

tstack:

‘Content’ property to ‘$group

because there is no specific way to write our contents($group) to in CSV parser/formatter. But we have it in JSON for-matter “Content” option.

Forum Discussion

Generate multiple files based on single column

6 Replies

Recent Discussions

Automating Untracked assets to GIT

Pagination and nextCursor in header

How to get filename from file reader

Generate expression file from database query

Can we generate XML file in pretty print format using native snapLogic snaps?