Forum Discussion

alex_panganiban's avatar
alex_panganiban
Contributor
7 years ago

Create Array from Flattened Docs

I could really use some help. I have flattened documents and for each distinct product-id value, I wish to create an array list of the variants that belong in the product-id’s family. My desired results should look something like this, but I’m not sure how to get there.

{

"masterproducts": [

{

"product-id": "1",

"variants": [

{

"variant": "988221"

},

{

"variant": "988331"

}]

},

{

"product-id": "1001",

"variants": [

{

"variant": "013356"

},

{

"variant": "933542"

},

{

"variant": "952005"

}]

},

{

"product-id": "1003",

"variants": [

{

"variant": "777100"

}]

}

]

}

 

Here's what I have so far.

 

My pipeline:

Output data from the sort snap. Sorting by product-id, variant.

What my GroupBy snap looks like:

13 Replies

  • Using Json generator I used this array of flattened docs:

    from there I sort on product-id using sort snap,

    then I groupBy product-id into a field called ‘group’:

    Finally in the mapper I use this expression:

    which generates an array of docs where each doc has ‘product-id’: [array of variants]:

    If you need the output to look EXACTLY like you described you can use this expression instead of the one above: { “product-id” : $group[0][‘product-id’],“variants” :$group.reduce((accum, curval) => accum.concat([curval[‘variant’]]), )}

    which will generate output exactly as you described: [{“product-id”:“1001”,“variants”:[“777101”,“777100”]},{“product-id”:“1002”,“variants”:[“777102”,“777103”,“777104”,“777105”]}]

    Let me know if this answers your problem.

  • Hi, I finally got it! Played around more with my expression and finally got what I needed. Sharing my solution:

    { “@product-id” : jsonPath($, “$group[0][‘@product-id’]”), “variations” : { “variants” : $group.reduce((accum, curval) => accum.concat([curval[‘variants’]]), ) } }

    • tstack's avatar
      tstack
      Former Employee

      Just a couple of notes… The jsonPath() call used in the first part shouldn’t be necessary, you can access the field using a plain expression, like so:

      $groupBy['@product-id']
      

      The jsonPath() function would actually be more useful for the second part since you can use a wildcard in the path to traverse the array and extract all values for a given field. So, instead of the reduce() call, you should be able to just do:

      jsonPath($, "$group[*].variant")
      
  • I am unable to see your pipeline or input data in those images. can you give a clear example of the input data and the desired output data for the given input?

    • alex_panganiban's avatar
      alex_panganiban
      Contributor

      thanks for responding. not sure why, but my images wouldn’t upload. i inquired about this and am currently waiting for a moderator to help me with getting my illustrations and examples uploaded.

    • alex_panganiban's avatar
      alex_panganiban
      Contributor

      Hi CJ, I was able to get the images uploaded. You can see what my input looks like, and what I’m trying to achieve, I’ve illustrated in the mocked up JSON that’s at the beginning of my post. Basically, for each product-id, I want there to be an array of variantid’s.

  • Oh, this is AWESOME! While the scenario I created isn’t exactly what I needed (I was trying to keep it simple), the solution you provided should be enough for me to run with. I’m gonna try it out now. Thanks so much for the assist!

    • cjhoward18's avatar
      cjhoward18
      Employee

      ok great, feel free to ask if you need further help.

      • alex_panganiban's avatar
        alex_panganiban
        Contributor

        Hi CJ, your solution has been really helpful to me. Thank you so much. I have an add-on question about this solution though. As it’s building the new array using the reduce statement you passed on to me, how can I insure that I’m not putting duplicate values into it? Hoping you can help.

        Thanks, Alex

  • Thanks so much again for your help. I’m pretty new to using both JSON and Javascript, so I’m learning quite q bit here. I’m so much closer to where I need to be.

    Ultimately, I need something that looks like this:

    Catalog
    Product : {
    @product-id : 1002,
    variations : {
    variants : [{
    variant @product-id : 888
    variant @product-id : 777
    variant @product-id : 666
    }]
    }
    }

    Output from my GroupBy:

    Expression Builder Expression and Map:


    My actual results. Rather than the “variant” element repeating, it seems to be looping back up to “variants.”

    I tried using a JSON Path Epresssion Tester that I found on Google, but it wasn’t much help for me. Most likely due to my inexperience. Any guidance would be greatly appreciated.

    Alex

  • Thought I’d let you know how I was able to remove duplicate entries from my array. I basically was using this reduce method to create my array in a mapper expression.

    $product.reduce((accum, curval) => accum.concat([curval[‘variation-attribute-value’][0]]), )

    To remove any duplicate entries while the array was being built, I added a filter method to it, like so:

    $product.reduce((accum, curval) => accum.concat([curval[‘variation-attribute-value’][0]]), ).filter((item, pos, a) => a.indexOf(item) == pos)

    I pretty much just copy/pasted this filter method from the Snaplogic help documentation.

    Thanks everyone for your help. I really appreciate all the support.