brianc0755
2 years agoNew Contributor II
diff snap usage in child pipeline
I use a diff snap to process records coming from a REST API call with records stored in a database. Records from the REST call are "New"; records from the database are "Original".
When the diff is in the parent pipeline everything works as expected.
When the diff is in a child pipeline and called with Pipeline Execute it does not. During the inital run there are no original records and all "New" records should be inserted; however, only one record is inserted and the rest are deleted. I have tried reusing existing connections, but that results in all records being treated as "New" each time the parent pipeline runs. Using a Gate or Group by N snap within the child pipeline causes a SQL error.