Converting value of input schema to distinct columns

[darshthakkar]

Hi Team,

I’m reading a text file and wanted to fetch DISTINCT columns out of the data itself, screenshot below for your reference.

.txt file:

When I read the above data in snaplogic, the format is:

Tried with $.merge(), GroupByN, CSV parse (contains headers) and Aggregrate (Concat) but didn’t find any luck.

Any suggestions over here?
Thank you.

Regards,
Darsh

[koryknick · Accepted solution]

Assuming it is always 8 properties, here is an example pipeline
Community 16940 - Converting value to distinct columns_2023_06_20.slp (6.3 KB)
Community-16940.txt (177 Bytes)

[darshthakkar]

Desired Output:

Name: Darsh
City: Toronto
Province-State: Ontario
Population: 126723

Country: Canada
IPV4: cc1d3198da257e12dffff507be1faeea
Phone: 70025416
Total Population: 126723

where Name, City, etc. are columns and Darsh, Toronto, etc. are values.

[koryknick]

If you can guarantee that these same 8 fields are always present, you could do a Group By N of 8, then process the array into an object. If missing values would remove the line, it becomes a bit more complex and may require a Script snap to handle the contextual parsing of the input.

[darshthakkar]

koryknick:

If you can guarantee that these same 8 fields are always present, you could do a Group By N of 8, then process the array into an object. If missing values would remove the line, it becomes a bit more complex and may require a Script snap to handle the contextual parsing of the input.

@koryknick - We can handle “n” no. of rows with Group Size as 0

I will modify my 50 to 0 now as 0 will handle all the incoming rows.

[koryknick · Accepted solution]

Assuming it is always 8 properties, here is an example pipeline
Community 16940 - Converting value to distinct columns_2023_06_20.slp (6.3 KB)
Community-16940.txt (177 Bytes)