Forum Discussion

shane's avatar
shane
New Contributor II
7 years ago
Solved

Join/Union missing data

I have 2 single field documents containing string representations of time (Ex: “12:03:43”) that when I use a Join snap with any join type, or when I use a Union, I see that there are items that don’t...
  • tlikarish's avatar
    tlikarish
    7 years ago

    Have you verified this is also the case when you execute the pipeline? One thing you might be hitting is the preview document limit. This is specified in the designer settings. The default is set to 50. Is your setting larger than the amount of data being input into the Join?

    What could be happening is that the Join is reading the first N documents on the left (upper) input. If 15:54:38 is after N, then it won’t appear in the preview output.

tlikarish's avatar
tlikarish
Employee
7 years ago

Have you verified this is also the case when you execute the pipeline? One thing you might be hitting is the preview document limit. This is specified in the designer settings. The default is set to 50. Is your setting larger than the amount of data being input into the Join?

What could be happening is that the Join is reading the first N documents on the left (upper) input. If 15:54:38 is after N, then it won’t appear in the preview output.

  • shane's avatar
    shane
    New Contributor II
    7 years ago

    This must be the issue.
    It looks like when I execute this doesn’t happen, but I was reluctant to do so when it appeared that data was being dropped.