How to use Deduplicate snap?

[mtran21]

I plan to use Deduplicate snap to remove duplicate records from a CSV file. In my CSV file, let’s say I have 3 records with the same ID. I just want to keep 1 record.

So I am using Deduplicate snap and set it up like this

It works but it also remove some of the not duplicated records. Why?

[akidave]

The thresholds in the snap default to values suited for approximate match. For exact match, you can set a higher threshold, like 0.9 and set a low criteria of 0.01 and high of 0.99