diff snap usage in child pipeline

[brianc0755]

I use a diff snap to process records coming from a REST API call with records stored in a database.  Records from the REST call are "New";  records from the database are "Original".   

When the diff is in the parent pipeline everything works as expected. 

When the diff is in a child pipeline and called with Pipeline Execute it does not.  During the inital run there are no original records and all "New" records should be inserted; however, only one record is inserted and the rest are deleted.  I have tried reusing existing connections, but that results in all records being treated as "New" each time the parent pipeline runs.  Using a Gate or Group by N snap within the child pipeline causes a SQL error.

[svatada]

@brianc0755  It could be related to a parent and child pipeline design issue. Check how your child pipeline is receiving documents from the parent pipeline.

To debug this issue, first have your child pipeline store all incoming records in a file. Then, use this file as input to the child pipeline and verify whether it processes the data correctly or not.