Finding alternative for Join/Union Snap?

As per my project, we are NOT ALLOWED to use join/union snap in our pipelines as according to their theory, it reduces the speed of the pipeline.

Can someone help me in finding the alternative?

Thanks in Advance

Hey @aditya.gupta41 ,

you can use the Gate snap and remap the data afterwards, but I seriously doubt that this will be faster than Union. You can also use In-Memory lookup, depending on what type of join do you need.

It is incorrect to say Union Snap “reduces the speed of the pipeline”. Union Snap is one of the simplest and fastest Snaps.
Join Snap is also very fast if the input data stream is sorted. If not, Join Snap’s internal sorter sorts and stages all input data in local temp files until input data streams reach the end. That is why users may see it reducing the speed. However, it is one of the most widely-used Snaps with a high price performance. If all your input data is already in a database, of course, it would be faster to join it in the database.
If you need to do inner-join just two input data streams and one of them is less than approximately 100,000 Map objects, In-Memory Lookup Snap would be faster than Join Snap since it stages the right view input data in memory.
If you post why your team does not allow to use Join/Union Snaps and why they think Join/Union Snaps are slow in more details, it would be easier to help you find alternatives.

3 Likes