I am working on this pipeline
The problem is with this part
I copy my result set.
I keep my first output
And aggregate my second output
And finally I want to take those aggregate results using a lookup.
This make my pipeline run endlessly. And if I remove the lookup (or join) and write to two different files it takes less than 2 minutes.
I think may be its because the two outputs are the image of the same result set.
Could you please tell me if you have seen this issue before and how to treat it.