Forum Discussion

jhagist's avatar
jhagist
New Contributor III
8 years ago
Solved

Rest API Test For Completion

I have a pattern where I run a REST POST api call and then need to run a REST GET to see if the request has completed. I need to somehow loop back to run the REST GET again if it isn’t done. Recom...
  • ptaylor's avatar
    ptaylor
    4 years ago

    Hi Wassim,

    This is actually a known issue (SWAT-3096) that we’re working on a fix for. It happens when there are at least 1024 records being copied by the Copy snap, for reasons that are a bit difficult to explain.

    Until we have a fix, there are at least three workarounds:

    • Swap the order of the inputs to the Lookup snap, so that the output of the Aggregate is the first input rather than the second.
    • Insert a Sort snap right after each output of the Copy snap. It won’t work if you put the Sort before the Copy. In this workaround, the point of the Sort snaps isn’t to sort the data, which might already be sorted – it’s to essentially create independent buffers of the data from each of the Copy snap’s output views.
    • Replace the Lookup with a Join, and set the Sorted streams property to Unsorted.
christwr's avatar
christwr
Contributor III
7 years ago
Copy LinkReport

I had that error too…

I was intrigued by the looping logic mentioned in the thread above because I had never seen or thought about doing something like that. But when I put together a simple pipeline to try and test it out (see below) I get the same StackOverflowError when trying to run or validate the pipeline.

As far as the snap it is failing on, I think it just mentions the first snap, but really failed trying to prepare the pipeline, not necessarily with the first snap…

cstewart's avatar
cstewart
Former Employee
7 years ago
Copy LinkReport

The way to address this one is to use the REST GET Snap, and use the HAS NEXT functionality. You can make it iterate until the correct response is received.
(You may not have a loop in a pipeline)

  • Szymon's avatar
    Szymon
    New Contributor II
    5 years ago
    Copy LinkReport

    Hi,

    I have tested a bit more. If I force an error in e.g. Mapper3 or LogMessage (see picture below), I see the child-error-pipeline getting executed fine. Only, when I force an error in JSON Formatter Snap, it fails with the error message mentioned in the initial post.
    Is it a BUG or a FEATURE, that JSON Formatter does not call error pipeline properly? Are there other snaps with this FEATURE?

    I also noticed, that the child-error-pipeline only gets executed once, for all the errors in the pipeline, WOW-1!
    What is more, even if errors occured in more snaps, pipeline gets executed further, and I see “Completed” status in the Dashboard, WOW-2! ← this probably is why, error pipeline gets executed only once. Is there a possibility to force the pipeline to stop on error and use error pipeline?

    cheers
    Simon 🙂