Join preview data different than the actual raw data

Designing Pipelines
, ,
#1

Hi Team,

I’m joining 2 tables (one from snowflake and another from salesforce), I do have the raw data so that it helps me with my validation while designing a pipeline on snapLogic.

While I use a left outer join, even though the data should have been populated in the output preview, it doesn’t whereas when I exported the same data in an excel file, it did show me the actual data! What would be the reason behind this one?

Just for everyone’s reference, I did validate the pipeline with Shift + Enter a couple of times but the result is the same. The challenge is the result I get out of my first join needs to be used for another join and as the result set is coming as “NULL” for the data that should have some value, further joins cannot be effectively used.

Any help on this would be highly appreciated, I may be doing something wrong with the snap settings and I’m more than happy to incorporate that feedback and get this up and running.

Thanking in advance for your time.

Best Regards,
DT

#2

HI @darshthakkar
Did you try executing pipeline upto that point and write data to file.
you can increase Preview Document Count to 2000 this might help to check data during validation.

Regards,
Ajay

#3

@Ajay_Chawda, Yes I did execute the pipeline but the preview generated is not similar to the data expected. Rationale behind this statement:

As I’m joining 2 tables, the primary key is an ID for both the tables, for instance, ID = 100 and Name (to be fetched field) is “Darsh”, preview would show Null although we know ID = 100 has a value for Name as Darsh, I’m confident with this as I have the raw data wherein I can do the validations.