Converting value of input schema to distinct columns

Question

Hi Team,
I’m reading a text file and wanted to fetch DISTINCT columns out of the data itself, screenshot below for your reference.
.txt file:

When I read the above data in snaplogic, the format is:

Tried with $.merge(), GroupByN, CSV parse (contains headers) and Aggregrate (Concat) but didn’t find any luck.
Any suggestions over here?
Thank you.
Regards,
Darsh

koryknick · Answer

If you can guarantee that these same 8 fields are always present, you could do a Group By N of 8, then process the array into an object.  If missing values would remove the line, it becomes a bit more complex and may require a Script snap to handle the contextual parsing of the input.

koryknick · Answer

@darshthakkar  - the reason I was asking about the number of fields is that the Group By N is giving me the set of fields per object.  If you increase that to consume the entire document, then creating the object would fail because you wind up with duplicate names in the object.
But I’m glad this is working for you!

koryknick · Answer

There are a handful of snaps that collect documents into fewer, such as:

Group By N
Group By Gate
Gate
Aggregate
Document to Binary
 Formatter snaps

Each of these serve a different purpose and output differently.  Some words of caution on the Document to Binary, Group By, and Gate snaps - These gather the complete input document set into an array for output.  So if you have large documents and gather all (or large number) of them into a single document, you can theoretically run your JCC out of memory and cause a failure.
With that said, with your original question, I assumed that the “set of 8” fields you had shown would be repeated in the incoming file / feed.  Assuming that the feed is non-repeating, then the Gate or Group By N set at 0 is an acceptable solution.  But if the fields repeat within the feed, you will get an error when a field of the same name is encountered.
Hope that helps!

darshthakkar · Answer

Desired Output:
Name: Darsh
City: Toronto
Province-State: Ontario
Population: 126723
Country: Canada
IPV4: cc1d3198da257e12dffff507be1faeea
Phone: 70025416
Total Population: 126723
where Name, City, etc. are columns and Darsh, Toronto, etc. are values.

darshthakkar · Answer

Got it, thank you for sharing the additional details on this.

Forum Discussion

Converting value of input schema to distinct columns

10 Replies

Recent Discussions

Javascript to promote top level lists

Google Sheets Subscribe questions

Basic string transformations not working

Can we generate XML file in pretty print format using native snapLogic snaps?

Multipart Reader failure - 'content-type' was not found