SnapLogic - Integration Nation

darshthakkar · ‎06-20-2023

Hi Team,

I’m reading a text file and wanted to fetch DISTINCT columns out of the data itself, screenshot below for your reference.

.txt file:

When I read the above data in snaplogic, the format is:

Tried with $.merge(), GroupByN, CSV parse (contains headers) and Aggregrate (Concat) but didn’t find any luck.

Any suggestions over here?
Thank you.

Regards,
Darsh

koryknick · ‎06-20-2023

Assuming it is always 8 properties, here is an example pipeline
Community 16940 - Converting value to distinct columns_2023_06_20.slp (6.3 KB)
Community-16940.txt (177 Bytes)

View solution in original post

darshthakkar · ‎06-21-2023

Thank you @koryknick, the solution worked.
I happen to tweak the expression in the mapper a bit so that no extra spaces were observed:

Yes, the text file will always have “x” fields, it wouldn’t even go to x-1 or x+1.

Moreover, as there is no max value in a group size as per documentation, I can pretty much keep it to 100 for instance to handle the additional rows coming in the future. What are your thoughts?
Intentionally, kept 100 and not 99999999 as I’m not anticipating anything more than 8.

Thank you.

Regards,
Darsh

View solution in original post

darshthakkar · ‎06-22-2023

Thank you for sharing the additional details, @koryknick.
Yup, my feed will only have 8 rows so all good over there but I was exploring the documentation, thought of bringing it to your notice so that you can help me understand it better.

As always, thank you so much for the additional details and tips.

SnapLogic - Integration Nation

Converting value of input schema to distinct columns