Forum Discussion

Harsha3's avatar
Harsha3
New Contributor III
5 years ago

RowVal in snaplogic

Hi ,

I have designed a pipeline where I need to apply duplication on specific fields instead of all to drive other fields . We have RowVal in oracle which does this job by including RowVal=1 . Can we implement the same in snaplogic . I give a sample file extract and the requirement .

----Sample

Where the count is assigned based on Call_ID , Product_ID, Is_Deleted
combination . In the screenshot we have the value of 2 in the count because the entry is duplicated . If we have the count value like this we can easily take the de-duplicate values by filtering $Count==1 . Please guide/propose a method which solves this

Thanks

4 Replies

  • j_angelevski's avatar
    j_angelevski
    Contributor III
    5 years ago

    I was able to achieve the desired result like this:
    I have sample data like this:

    Then I use Group by Field snap

    Then a Mapper with the following expression jsonPath($, "object[*]").map((val, index, arr) => val.extend({"Count": index == 0 ? 1 : 2})). This way you are adding a Count field to every object, if index is 0 meaning first object then add a count of 1, if index > 0 then Count will always be 2. Output:

    Then you can use filter snap to filter out those records with count = 2.
    Make sure you add a sort by snap before group by snap.

    Edit:
    Also, if you don’t need the Count field you can make it more simple with a mapper with the following expression jsonPath($, "object[*]")[0], the output will be the same as above except there is no Count field. This way you only get one object from the grouped records (meaning all other duplicates are ignored).

    • Harsha3's avatar
      Harsha3
      New Contributor III
      5 years ago

      Thanks so much Angelevski , this approach worked fine

  • j_angelevski's avatar
    j_angelevski
    Contributor III
    5 years ago

    Hi @Harsha3,

    As far as I understood, you want to create the “Count” field in snaplogic based on Call_ID, Product_ID, Is_Deleted. Meaning if those values are the same you want to make count = 2 ? And then filter out those with count = 2 ?

    • Harsha3's avatar
      Harsha3
      New Contributor III
      5 years ago

      Hi Angelevski,

      Exactly it has to be like that . I have a sample data here . I tried groupBy snaps and not able to get the desired result.

      Thanks,
      Harshavardhana