Filter function in snaplogic

[Anjali]

Can anyone make me understand what the below function is doing?

filter((a, b, c) => c.indexOf(a) == b)

[RogerSramkoski]

The filter() function is taking (a, b, c) as parameters and everything after => is an “on the fly” function to help determine which values from (a, b, c) should be in the output. If I’m not mistaken, the function is searching string ‘c’ to see if it contains string ‘a’, and if it so, does that equal integer ‘b’. Basically, if string ‘a’ is detected inside string ‘c’ at the expected location ‘b’, then the document would pass through the filter, otherwise it would be excluded.

In SnapLogic you can use filter() on Objects and Arrays. If you’re able to share more context around the expression such as sample data, or all/part of a pipeline that uses it, we can try to get you more information.

Please let us know if that helps and if some part still needs clarified, please let us know.

[Anjali]

To remove duplicate entries in an array:

Expression : $myarray.filter((item, pos, a) => a.indexOf(item) == pos)

where $myarray is a normal array containing [“Fred”, “Wilma”, “Fred”, “Betty”, “Fred”, “Barney”]

Result : A new array containing [Fred, Wilma, Betty, Barney].

I found this example in documentation. but not able to understand how this function is deleting duplicates. Can you explain with this example

[Anjali]

I got my answer how this filter function deletes duplicates

JavaScript Tutorial – 13 Jan 20

JavaScript: Remove Duplicates from an Array

In this tutorial, you will learn some techniques to to remove duplicates from an array in JavaScript.

Est. reading time: 6 minutes

[RogerSramkoski]

Thank you for sharing the article that helped you. I took a look at it and I can definitely see the author did a great job going step-by-step, explaining how it works along the way. For anyone else looking to understand what is happening with the specific SnapLogic example, here is the explanation.

First, let’s just restate the data and expression we’re working with:
Data: $myarray = [“Fred”, “Wilma”, “Fred”, “Betty”, “Fred”, “Barney”]
Expression: $myarray.filter((item, pos, a) => a.indexOf(item) == pos)

At a high level, the filter() function is going to be passed three pieces of information:

• item is the element value like “Fred” or “Wilma”
• pos is the element’s position in the array starting with 0 for the first element
• a is the array

Now let’s take a look at the first use of the function and the values we’re working with. Note the value of a will always be the full array.

• item = “Fred”
• pos = 0
• a = [“Fred”, “Wilma”, “Fred”, “Betty”, “Fred”, “Barney”]

So on the first past we are searching the array for “Fred” and indexOf() will return 0, which matches the value of pos, meaning it is the first occurrence. The filter() function would then move “Fred” through to the resulting array. The second pass plays out the same situation but this time our values look like this:

• item = “Wilma”
• pos = 1
• a = [“Fred”, “Wilma”, “Fred”, “Betty”, “Fred”, “Barney”]

As indexOf() iterates through the array the first occurrence of “Wilma” is index 1, which matches our pos, so filter() passes “Wilma” through to the resulting array which would now look like [“Fred”, “Wilma”]. On the third pass is when we observe filter() knock out a duplicate item from the array. Let’s take a look at the values:

• item = “Fred”
• pos = 2
• a = [“Fred”, “Wilma”, “Fred”, “Betty”, “Fred”, “Barney”]

For the third pass, the indexOf(“Fred”) result would be 0, which does not equal our pos of 2, meaning that this item has already occurred in the array, therefore filter() will not pass it through.

Hope that helps give a clearer picture of how that expression works specifically with the SnapLogic example! If you’re looking for an example pipeline to test this with you can find one in our Array.filter() documentation here.