01-06-2021 07:33 PM
Hi, Below is my pipeline where iam trying to upload data from oracle execute query which returns 200k records. iam grouping them into 10k at a time to Rest Data Upload snap(rest post).
My question - is there a way i can stay on Rest Data Upload snap until all 200k records load.
Tried using script snap to sleep for 10min but for some reason, script snap getting triggered after first 10k records and going to subsequent snaps.
01-06-2021 07:47 PM
You should use a Gate snap, or the group by (0) snap after the rest upload, which will wait for the completion of the stream.