I have 2 single field documents containing string representations of time (Ex: “12:03:43”) that when I use a Join snap with any join type, or when I use a Union, I see that there are items that don’t make it in the join.
I have tried various things and no matter how I join these two documents containing ‘time’ strings there are data-points dropped in the join.
I also have seen simple copy snaps drop data as well. I don’t have screenshots of this but it seems that running it through a ‘Unique’ snap resolved that issue, although I don’t know why that should matter.
1 ACCEPTED SOLUTION
Have you verified this is also the case when you execute the pipeline? One thing you might be hitting is the preview document limit. This is specified in the designer settings. The default is set to 50. Is your setting larger than the amount of data being input into the Join?
What could be happening is that the Join is reading the first N documents on the left (upper) input. If 15:54:38 is after N, then it won’t appear in the preview output.
Have you verified this is also the case when you execute the pipeline? One thing you might be hitting is the preview document limit. This is specified in the designer settings. The default is set to 50. Is your setting larger than the amount of data being input into the Join?
What could be happening is that the Join is reading the first N documents on the left (upper) input. If 15:54:38 is after N, then it won’t appear in the preview output.
