REST pagination works in validate but fails in executions

[dgalas]

Hi, I’m having an issue running a pipeline with a REST GET with a next URL. When I run a validation, the SNAP works fine, and I even see multiple responses indicating the next URL seems to be returned. When I run an execution however, the SNAP aborts and I get the following error:

“com.snaplogic.snap.api.SnapDataException: REST API service endpoint returned error result: status code = 200
at com.snaplogic.snap.api.rest.RestResponseProcessor.process(RestResponseProcessor.java:336)
at com.snaplogic.snap.api.rest.RestRequestExecutor.execute(RestRequestExecutor.java:369)
at com.snaplogic.snap.api.rest.RestRequestExecutor.executeOnce(RestRequestExecutor.java:303)
at com.snaplogic.snap.api.rest.RestRequestExecutor.execute(RestRequestExecutor.java:186)
at com.snaplogic.snap.api.rest.RestRequestExecutor.execute(RestRequestExecutor.java:169)
at com.snaplogic.snaps.rest.Get.process(Get.java:147)
at com.snaplogic.snap.api.ExecutionUtil.process(ExecutionUtil.java:106)
at com.snaplogic.snap.api.ExecutionUtil.executeForDocument(ExecutionUtil.java:118)
at com.snaplogic.snap.api.ExecutionUtil.execute(ExecutionUtil.java:81)
at com.snaplogic.snap.api.SimpleSnap.execute(SimpleSnap.java:70)
at com.snaplogic.cc.snap.common.SnapRunnableImpl.executeSnap(SnapRunnableImpl.java:803)
at com.snaplogic.cc.snap.common.SnapRunnableImpl.execute(SnapRunnableImpl.java:585)
at com.snaplogic.cc.snap.common.SnapRunnableImpl.doRun(SnapRunnableImpl.java:869)
at com.snaplogic.cc.snap.common.SnapRunnableImpl.call(SnapRunnableImpl.java:435)
at com.snaplogic.cc.snap.common.SnapRunnableImpl.call(SnapRunnableImpl.java:117)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: com.snaplogic.cc.snap.common.SnapStreamException: Exception while writing document to the stream
at com.snaplogic.cc.snap.view.document.DocumentOutputViewImpl.write(DocumentOutputViewImpl.java:251)
at com.snaplogic.snap.api.OutBoundViewsImpl.write(OutBoundViewsImpl.java:221)
at com.snaplogic.snap.api.OutBoundViewsImpl.write(OutBoundViewsImpl.java:257)
at com.snaplogic.snap.api.rest.RestResponseProcessor.writeToOutput(RestResponseProcessor.java:630)
at com.snaplogic.snap.api.rest.RestResponseProcessor.processEntity(RestResponseProcessor.java:659)
at com.snaplogic.snap.api.rest.RestResponseProcessor.process(RestResponseProcessor.java:316)
… 20 more
Caused by: com.snaplogic.cc.jstream.AbortException: Attempt to write to closed view
at com.snaplogic.cc.jstream.view.publisher.NullPublisher.publish(NullPublisher.java:53)
at com.snaplogic.cc.snap.view.document.DocumentOutputViewImpl.lambda$write$0(DocumentOutputViewImpl.java:224)
at com.snaplogic.cc.snap.common.ViewImpl.timeCall(ViewImpl.java:658)
at com.snaplogic.cc.snap.view.document.DocumentOutputViewImpl.write(DocumentOutputViewImpl.java:156)
… 25 more
Reason: Error occurred while parsing JSON :Attempt to write to closed view
Resolution: Please check the headers of the request and make sure that json is returned by the endpoint.”

I don’t really understand what is happening, but I’m guessing the output view is closing before the snap has finished? Is there any advice on how I can resolve this?

Thank you.

[dgalas]

I think I figured out my issue. The documentation for the API I was calling mentioned that if a value in the response was null that there weren’t any more records, but the response value was actually an empty object, so my HasNext was essentially always true, which seemed to cause the snap to error out.