RowVal in snaplogic

[Harsha3]

Hi ,

I have designed a pipeline where I need to apply duplication on specific fields instead of all to drive other fields . We have RowVal in oracle which does this job by including RowVal=1 . Can we implement the same in snaplogic . I give a sample file extract and the requirement .

----Sample

Where the count is assigned based on Call_ID , Product_ID, Is_Deleted
combination . In the screenshot we have the value of 2 in the count because the entry is duplicated . If we have the count value like this we can easily take the de-duplicate values by filtering $Count==1 . Please guide/propose a method which solves this

Thanks

[j_angelevski]

Hi @Harsha3,

As far as I understood, you want to create the “Count” field in snaplogic based on Call_ID, Product_ID, Is_Deleted. Meaning if those values are the same you want to make count = 2 ? And then filter out those with count = 2 ?

[Harsha3]

Hi Angelevski,

Exactly it has to be like that . I have a sample data here . I tried groupBy snaps and not able to get the desired result.

Thanks,
Harshavardhana

[j_angelevski]

I was able to achieve the desired result like this:
I have sample data like this:

Then I use Group by Field snap

Then a Mapper with the following expression jsonPath($, "object[*]").map((val, index, arr) => val.extend({"Count": index == 0 ? 1 : 2})). This way you are adding a Count field to every object, if index is 0 meaning first object then add a count of 1, if index > 0 then Count will always be 2. Output:

Then you can use filter snap to filter out those records with count = 2.
Make sure you add a sort by snap before group by snap.

---

Edit:
Also, if you don’t need the Count field you can make it more simple with a mapper with the following expression jsonPath($, "object[*]")[0], the output will be the same as above except there is no Count field. This way you only get one object from the grouped records (meaning all other duplicates are ignored).

[Harsha3]

Thanks so much Angelevski , this approach worked fine