Create Array from Flattened Docs

I could really use some help. I have flattened documents and for each distinct product-id value, I wish to create an array list of the variants that belong in the product-id’s family. My desired results should look something like this, but I’m not sure how to get there.

{

"masterproducts": [

{

"product-id": "1",

"variants": [

{

"variant": "988221"

},

{

"variant": "988331"

}]

},

{

"product-id": "1001",

"variants": [

{

"variant": "013356"

},

{

"variant": "933542"

},

{

"variant": "952005"

}]

},

{

"product-id": "1003",

"variants": [

{

"variant": "777100"

}]

}

]

}

 

Here's what I have so far.

 

My pipeline:

Output data from the sort snap. Sorting by product-id, variant.

image

What my GroupBy snap looks like:

image003

I am unable to see your pipeline or input data in those images. can you give a clear example of the input data and the desired output data for the given input?

thanks for responding. not sure why, but my images wouldn’t upload. i inquired about this and am currently waiting for a moderator to help me with getting my illustrations and examples uploaded.

Hi CJ, I was able to get the images uploaded. You can see what my input looks like, and what I’m trying to achieve, I’ve illustrated in the mocked up JSON that’s at the beginning of my post. Basically, for each product-id, I want there to be an array of variantid’s.

Using Json generator I used this array of flattened docs:

from there I sort on product-id using sort snap,

then I groupBy product-id into a field called ‘group’:

Finally in the mapper I use this expression:

which generates an array of docs where each doc has ‘product-id’: [array of variants]:

If you need the output to look EXACTLY like you described you can use this expression instead of the one above: { “product-id” : $group[0][‘product-id’],“variants” :$group.reduce((accum, curval) => accum.concat([curval[‘variant’]]), [])}

which will generate output exactly as you described: [{“product-id”:“1001”,“variants”:[“777101”,“777100”]},{“product-id”:“1002”,“variants”:[“777102”,“777103”,“777104”,“777105”]}]

Let me know if this answers your problem.

Oh, this is AWESOME! While the scenario I created isn’t exactly what I needed (I was trying to keep it simple), the solution you provided should be enough for me to run with. I’m gonna try it out now. Thanks so much for the assist!

ok great, feel free to ask if you need further help.

Thanks so much again for your help. I’m pretty new to using both JSON and Javascript, so I’m learning quite q bit here. I’m so much closer to where I need to be.

Ultimately, I need something that looks like this:

Catalog
Product : {
@product-id : 1002,
variations : {
variants : [{
variant @product-id : 888
variant @product-id : 777
variant @product-id : 666
}]
}
}

Output from my GroupBy:

Expression Builder Expression and Map:


My actual results. Rather than the “variant” element repeating, it seems to be looping back up to “variants.”

I tried using a JSON Path Epresssion Tester that I found on Google, but it wasn’t much help for me. Most likely due to my inexperience. Any guidance would be greatly appreciated.

Alex

Hi, I finally got it! Played around more with my expression and finally got what I needed. Sharing my solution:

{ “@product-id” : jsonPath($, “$group[0][’@product-id’]”), “variations” : { “variants” : $group.reduce((accum, curval) => accum.concat([curval[‘variants’]]), []) } }

1 Like

Just a couple of notes… The jsonPath() call used in the first part shouldn’t be necessary, you can access the field using a plain expression, like so:

$groupBy['@product-id']

The jsonPath() function would actually be more useful for the second part since you can use a wildcard in the path to traverse the array and extract all values for a given field. So, instead of the reduce() call, you should be able to just do:

jsonPath($, "$group[*].variant")
1 Like

Hi CJ, your solution has been really helpful to me. Thank you so much. I have an add-on question about this solution though. As it’s building the new array using the reduce statement you passed on to me, how can I insure that I’m not putting duplicate values into it? Hoping you can help.

Thanks, Alex

glad it was helpful.
You can use a map to store your values instead of an array and store the values as your keys. Maps do not allow for duplicate key entries, which would take care of this problem for you, and you could collect your ‘values array’ by using keys() method described here:
https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1439367/Object+Functions+and+Properties#ObjectFunctionsandProperties-keys

This is just one solution, there are many ways you can keep track of this if needed.

okay, i’ll try that out.

Thought I’d let you know how I was able to remove duplicate entries from my array. I basically was using this reduce method to create my array in a mapper expression.

$product.reduce((accum, curval) => accum.concat([curval[‘variation-attribute-value’][0]]), [])

To remove any duplicate entries while the array was being built, I added a filter method to it, like so:

$product.reduce((accum, curval) => accum.concat([curval[‘variation-attribute-value’][0]]), []).filter((item, pos, a) => a.indexOf(item) == pos)

I pretty much just copy/pasted this filter method from the Snaplogic help documentation.

Thanks everyone for your help. I really appreciate all the support.