Forum Discussion
@darshthakkar - as @ptaylor mentioned, when you are working with Pipeline Preview data and need to see data movement through the entire pipeline, you need to reduce your source data sets to a limited enough set that any snaps that can potentially filter data (i.e. Join, Filter, etc.) will still have data to pass. In your case, I would suggest putting a temporary filtering condition (i.e. Where Clause) on your database Select snap so that your data will match after the Join snap. Once you have verified that the pipeline logic is correct, simply remove the filter from your select and execute the entire pipeline and validate the complete results.
- darshthakkar4 years agoValued Contributor
Thank you @koryknick for your suggestions. I will follow this practice going forward so that I’m not stuck with my development.
I’ll be closing this thread soon listing the best practices and solution provided. As both @ptaylor and you were really instrumental in providing the suggestions, I wouldn’t be able to mark a solution for either of your comments but will be giving credits on my closing comment coming soon (may be end of the day today)
Thank you again both of you for your help on this one.
Regards,
Darsh