Forum Discussion

mohamadelmardin's avatar
mohamadelmardin
New Contributor III
9 years ago

How to render distinct elements only in an array with duplicates

I have a JSON document that has inside an array with duplicate values such as [1,1,3,3,6,1,2,1,2,100]. I am trying to transform/map it from source into target by returning only the distinct values such as [1,3,6,100]. How can I achieve that using a Mapper snap or if any other snap. For example when using jsonPath() function to specify a part of JSON document that has an array, is there a function similar to .unique() that could be attached to return only unique array elements?
The Unique snap works only on entire documents/rows not array elements within a document.

9 Replies

  • tstack's avatar
    tstack
    Former Employee

    The expression language is a subset of JavaScript, so you can usually start by looking for JavaScript-based solutions.

    In this case, take a look at this post:

    The first answer has the simplest solution. The only change you have to make is to convert the callback function to use the arrow-function syntax, like so:

    $arr.filter((item, pos, a) => a.indexOf(item) == pos)

    • srao's avatar
      srao
      New Contributor

      Hi, I have an complex JSON array and I have to remove the duplicates based on a field. For Ex:
      [ {PartnerID:1127839, ProductNum:a, …} ,
      {PartnerID:1127839, ProductNum:a,…} ,
      {PartnerID:1127839, ProductNum:b,…} ,
      {PartnerID:1127839, ProductNum:c,…} ],
      How can I remove the duplicate array elements based on the ProductNum Field to produce the below JSON
      [ {PartnerID:1127839, ProductNum:a, …} ,
      {PartnerID:1127839, ProductNum:b,…} ,
      {PartnerID:1127839, ProductNum:c,…} ]

      I have tried the above logic and it didnt work, as other fields in this json structure.

      • tstack's avatar
        tstack
        Former Employee

        Same idea, except you’d want to use the ‘findIndex()’ method with a custom predicate instead of the ‘indexOf()’ method, like so:

        $arr.filter((item, pos, a) => a.findIndex(elem => item.ProductNum == elem.ProductNum) == pos)
        

        So, the ‘findIndex()’ method will walk through the array and evaluate the predicate function (elem => item.ProductNum == elem.ProductNum) on every element in the array. If a match is found, the index is returned and, if it’s the same index as the current element being examined in the filter, then we add the element to output list.

    • Anjali's avatar
      Anjali
      New Contributor

      Can you please explain how this filter function is working? what is item , pos and a here and how there are functioning?