cancel
Showing results for 
Search instead for 
Did you mean: 

How do you keep only one record of duplicated records?

mtran21
New Contributor III

My pipeline is a simple pipeline that execute an Oracle query then parse the data to a CSV file. The result of the query gives some of the duplicate records (Same ID), that because the database has those duplicates. However, I just want to keep only 1 record of each duplicate rows. How do I do that?

image

6 REPLIES 6

okay, then you can use group by snap to group by ID and map required fields. attaching the sample pipeline and the data.
readCsv_2021_01_19.slp (7.6 KB) Screen Shot 2021-01-19 at 10.58.26 AM

@mtran21 Try use Duplicate snap and use column Id as Field .

image