Forum Discussion

amar's avatar
amar
New Contributor II
5 years ago

Hold Rest Post Snap until all records uploaded

Hi, Below is my pipeline where iam trying to upload data from oracle execute query which returns 200k records. iam grouping them into 10k at a time to Rest Data Upload snap(rest post).
My question - is there a way i can stay on Rest Data Upload snap until all 200k records load.
Tried using script snap to sleep for 10min but for some reason, script snap getting triggered after first 10k records and going to subsequent snaps.

pipeline

1 Reply

  • cstewart's avatar
    cstewart
    Former Employee
    5 years ago

    You should use a Gate snap, or the group by (0) snap after the rest upload, which will wait for the completion of the stream.