Issues using Diff Snap
Hi,
I’m new to snap logic, I have two data sources (Oracle, SQL) the new is Oracle and existing is SQL.
The pipeline’s first run, runs as expected, but the second, where I would expect all data ...
Forum Discussion
@dmcnichol
My guess is that you have large number of records in both of the sources, and when you hit execute, the pipeline works fine, detects modifications and unmodified rows accordingly. But when you try to validate the data, you are validating the pipeline, and by that, you end up with 2 completely different sets of data from both sources. Meaning, the IDs from the “original” set are not in the “new” set.
So as a result of that, all of the IDs from one source will go to Deletions, and from the other source, will go to Insertions.
Find a way to validate the data differently. One suggestion is to take specific group of IDs, and when you validate, you’ll make sure to use those IDs in both sources.
