11-10-2023 07:24 PM - edited 11-10-2023 07:32 PM
Please I am trying to understand the behavior and difference between these two snap set-ups. Any explanation will be appreciated. What exactly is the function of the tail snap? Thanks.
Solved! Go to Solution.
11-13-2023 04:47 AM
@omiaye - The Snowflake Insert snap is a transactional snap and will output one document for every document sent to it (assuming successful insert to the database). So if you have 700 records in, you get 700 records out. In your first example, this means the Snowflake execute will be called 700 times, so if it is a query that is returning 700 records, then you get the 700 * 700 documents out (49k).
In the second example, you are using the Tail snap, which by definition: ..."reads the specified number of documents from the end of the input stream." In other words, if you have 700 documents in and set the "Number of documents" in the Tail snap to 1 (default), it returns only the last record. This is a great way to wait for all upstream documents to be delivered before continuing to the next step... in your case, the Snowflake Execute snap.
Hope this helps!
11-13-2023 04:47 AM
@omiaye - The Snowflake Insert snap is a transactional snap and will output one document for every document sent to it (assuming successful insert to the database). So if you have 700 records in, you get 700 records out. In your first example, this means the Snowflake execute will be called 700 times, so if it is a query that is returning 700 records, then you get the 700 * 700 documents out (49k).
In the second example, you are using the Tail snap, which by definition: ..."reads the specified number of documents from the end of the input stream." In other words, if you have 700 documents in and set the "Number of documents" in the Tail snap to 1 (default), it returns only the last record. This is a great way to wait for all upstream documents to be delivered before continuing to the next step... in your case, the Snowflake Execute snap.
Hope this helps!
11-13-2023 05:52 AM
Thank you so much @koryknick.