Sort snaps takes more time

[Sowmya_Rayavara]

Hi All,

I have four sort snaps which run parallel. Few times they are getting sorted very quickly(in seconds), especially first time when I ran the interface, but next time or second time onwards, it is taking more time almost few hours. On the next day when I ran the interface again it is completed in few seconds. Could anyone please help me to understand what is happening? Also there is no change in the interface and also there are no other interfaces running in the servers.

Regards,
Sowmya.

[shane]

The following snap is just a Unique snap which results (in that specific instance) with the same result as was fed in.

After that is a simple JDBC Select snap that when I check query logs, the select queries took ~2.5seconds each, and were pending for about 3 mins prior to execution but that snap seems to have taken 24 mins to execute. I sat there waiting for these snaps to finish for about 20 mins after the queries completed according to my redshift query logs.
Either way it seems to just hang there and I can’t determine what is causing the hang.

This does seem to be getting into something not quite related to the topic though lol. I’ll have to dig in deeper but unfortunately I can’t see every query in the Redshift query logs (could be part of my issue). This time upon execution of the pipeline the Redshift query logs showed no select statements at all.

[tstack]

shane:

The following snap is just a Unique snap

What about the snaps that follow that, are there any with a long blue bar in the duration column that would indicate a snap that is taking a long time to execute? Maybe include a screenshot of the execution statistics. The Execution Statistics documentation might help you understand the details.