Sort snaps takes more time
Hi All,
I have four sort snaps which run parallel. Few times they are getting sorted very quickly(in seconds), especially first time when I ran the interface, but next time or second time onwards, ...
Forum Discussion
What is the snap downstream from the Sort? The “Output time” statistic there shows that the Sort snap seems to be waiting on the downstream snap to do its work. The time taken to do the Sort was less than a second.
The following snap is just a Unique snap which results (in that specific instance) with the same result as was fed in.
After that is a simple JDBC Select snap that when I check query logs, the select queries took ~2.5seconds each, and were pending for about 3 mins prior to execution but that snap seems to have taken 24 mins to execute. I sat there waiting for these snaps to finish for about 20 mins after the queries completed according to my redshift query logs.
Either way it seems to just hang there and I can’t determine what is causing the hang.
This does seem to be getting into something not quite related to the topic though lol. I’ll have to dig in deeper but unfortunately I can’t see every query in the Redshift query logs (could be part of my issue). This time upon execution of the pipeline the Redshift query logs showed no select statements at all.
What about the snaps that follow that, are there any with a long blue bar in the duration column that would indicate a snap that is taking a long time to execute? Maybe include a screenshot of the execution statistics. The Execution Statistics documentation might help you understand the details.
