Forum Discussion

Ish's avatar
Ish
New Contributor III
4 years ago
Solved

REST Get delay between making calls as documents come in (not pagination or retry)

Hello,

For the REST Get snap, there’s a feature to set a delay interval between Pagination and Retry calls, but what about just between regular calls as documents come in? All calls to REST can be rate limited, not just pagination calls. Do I have to create a sub-pipeline with a script delay to achieve this?

Thanks!

REST
  • j_angelevski's avatar
    j_angelevski
    4 years ago

    Hi @Ish,

    If you really need the delay between each REST call on each document input, then yes that has to be delayed with a Script snap, it doesn’t have to in a sub-pipeline because you can pass the data through the Script snap.

5 Replies

  • Ish's avatar
    Ish
    New Contributor III
    4 years ago

    Another thing to mention – sometimes REST APIs tell you how long you have to wait until you can make the next call. It would be great if the REST Get could be enhanced to not only allow setting a delay, but also the ability to use the results from the previous call to set how long the delay will be before the next one.

  • rgoerwit's avatar
    rgoerwit
    New Contributor II
    4 years ago

    This is an ongoing point of pain, and many folks have requested the ability to loop in SL without resorting to deep recursion.

    There are apparently technical barriers to this. But it’s necessary. (Loop here would be: GET, examine response; if response says to wait, extract the wait time and pause, then go back to step one; otherwise, proceed.)

    • joel_bourgault's avatar
      joel_bourgault
      New Contributor III
      2 years ago

      Interesting use case; it seems to me that if Pagination Interval was a field that can be computed, it would implement the expected behavior.

  • j_angelevski's avatar
    j_angelevski
    Contributor III
    4 years ago

    Hi @Ish,

    If you really need the delay between each REST call on each document input, then yes that has to be delayed with a Script snap, it doesn’t have to in a sub-pipeline because you can pass the data through the Script snap.

    • Ish's avatar
      Ish
      New Contributor III
      4 years ago

      Thank you for the tip – I’ll implement it this way.