REST Get Snap fails with Reason: Error occurred while parsing JSON :Premature end of chunk coded message body: closing chunk expected

[Prasa]

Can anyone help me how to solve this issue.
We are pulling full load data from Iterable daily and it takes around 6-7 hours to complete 20 million of records. Most of the times the pipeline fails with below reason after running for 1- 2hours.
Please help me to resolve this issue.
Stacktrace-

REST API service endpoint returned error result: status code = 200

Resolution:

Please check the headers of the request and make sure that json is returned by the endpoint.

Reason:

Error occurred while parsing JSON :Premature end of chunk coded message body: closing chunk expected

Hide Details…

All_Export Users from Iterable-Marketing-FullLoad
`com.snaplogic.snap.api.SnapDataException: REST API service endpoint returned error result: status code = 200
at com.snaplogic.snap.api.rest.RestResponseProcessor.process(RestResponseProcessor.java:349)
at com.snaplogic.snap.api.rest.RestRequestExecutor.execute(RestRequestExecutor.java:372)
at com.snaplogic.snap.api.rest.RestRequestExecutor.executeOnce(RestRequestExecutor.java:303)
at com.snaplogic.snap.api.rest.RestRequestExecutor.execute(RestRequestExecutor.java:186)
at com.snaplogic.snap.api.rest.RestRequestExecutor.execute(RestRequestExecutor.java:169)
at com.snaplogic.snaps.rest.Get.process(Get.java:124)
at com.snaplogic.snap.api.ExecutionUtil.process(ExecutionUtil.java:106)
at com.snaplogic.snap.api.ExecutionUtil.executeForDocument(ExecutionUtil.java:118)
at com.snaplogic.snap.api.ExecutionUtil.execute(ExecutionUtil.java:81)
at com.snaplogic.snap.api.SimpleSnap.execute(SimpleSnap.java:70)
at com.snaplogic.cc.snap.common.SnapRunnableImpl.executeSnap(SnapRunnableImpl.java:803)
at com.snaplogic.cc.snap.common.SnapRunnableImpl.execute(SnapRunnableImpl.java:585)
at com.snaplogic.cc.snap.common.SnapRunnableImpl.doRun(SnapRunnableImpl.java:869)
at com.snaplogic.cc.snap.common.SnapRunnableImpl.call(SnapRunnableImpl.java:435)
at com.snaplogic.cc.snap.common.SnapRunnableImpl.call(SnapRunnableImpl.java:117)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: com.snaplogic.api.ExecutionException: Unable to parse JSON input
at com.snaplogic.snap.api.read.AbstractJsonProvider.hasNext(AbstractJsonProvider.java:87)
at com.snaplogic.snap.api.rest.RestResponseStreamProcessor.processEntity(RestResponseStreamProcessor.java:55)
at com.snaplogic.snap.api.rest.RestResponseProcessor.process(RestResponseProcessor.java:329)
… 20 more
Caused by: java.lang.RuntimeException: Premature end of chunk coded message body: closing chunk expected
at com.fasterxml.jackson.databind.MappingIterator._handleIOException(MappingIterator.java:417)
at com.fasterxml.jackson.databind.MappingIterator.hasNext(MappingIterator.java:190)
at com.snaplogic.snap.api.read.AbstractJsonProvider.hasNext(AbstractJsonProvider.java:85)
… 22 more
Caused by: org.apache.http.ConnectionClosedException: Premature end of chunk coded message body: closing chunk expected
at org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:263)
at org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:222)
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:183)
at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._loadMore(UTF8StreamJsonParser.java:220)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._skipWSOrEnd2(UTF8StreamJsonParser.java:3036)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._skipWSOrEnd(UTF8StreamJsonParser.java:3031)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:719)
at com.fasterxml.jackson.databind.MappingIterator.hasNextValue(MappingIterator.java:243)
at com.fasterxml.jackson.databind.MappingIterator.hasNext(MappingIterator.java:186)
… 23 more
Reason: Error occurred while parsing JSON :Premature end of chunk coded message body: closing chunk expected
Resolution: Please check the headers of the request and make sure that json is returned by the endpoint.

Error Fingerprint[0] = efp:com.snaplogic.snap.api.rest.KNOIW-k6
Error Fingerprint[1] = efp:com.snaplogic.snap.api.read.zx5ZIG2f
Error Fingerprint[2] = efp:com.fasterxml.jackson.databind.v1RgvKSD
Error Fingerprint[3] = efp:org.apache.http.impl.io.S-JRxMrP`

All_Export Users from Iterable-Marketing-FullLoad
`com.snaplogic.cc.snap.common.ThreadDetails: prio=4 Id=3106061 RUNNABLE
at java.base@11.0.8/java.util.stream.AbstractPipeline.wrapSink(Unknown Source)
at java.base@11.0.8/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
at java.base@11.0.8/java.util.stream.FindOps$FindOp.evaluateSequential(Unknown Source)
at java.base@11.0.8/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
at java.base@11.0.8/java.util.stream.ReferencePipeline.findFirst(Unknown Source)
at org.apache.logging.log4j.util.StackLocator.lambda$calcLocation$15(StackLocator.java:99)
at org.apache.logging.log4j.util.StackLocator$Lambda$114/0x00000008000b3040.apply(Unknown Source)
at java.base@11.0.8/java.lang.StackStreamFactory$StackFrameTraverser.consumeFrames(Unknown Source)
…
at java.base@11.0.8/java.util.stream.AbstractPipeline.wrapSink(Unknown Source)
at java.base@11.0.8/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
at java.base@11.0.8/java.util.stream.FindOps$FindOp.evaluateSequential(Unknown Source)
at java.base@11.0.8/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
at java.base@11.0.8/java.util.stream.ReferencePipeline.findFirst(Unknown Source)
at org.apache.logging.log4j.util.StackLocator.lambda$calcLocation$15(StackLocator.java:99)
at org.apache.logging.log4j.util.StackLocator$Lambda$114/0x00000008000b3040.apply(Unknown Source)
at java.base@11.0.8/java.lang.StackStreamFactory$StackFrameTraverser.consumeFrames(Unknown Source)
at java.base@11.0.8/java.lang.StackStreamFactory$AbstractStackWalker.doStackWalk(Unknown Source)
at java.base@11.0.8/java.lang.StackStreamFactory$AbstractStackWalker.callStackWalk(Native Method)
at java.base@11.0.8/java.lang.StackStreamFactory$AbstractStackWalker.beginStackWalk(Unknown Source)
at java.base@11.0.8/java.lang.StackStreamFactory$AbstractStackWalker.walk(Unknown Source)
at java.base@11.0.8/java.lang.StackWalker.walk(Unknown Source)
at org.apache.logging.log4j.util.StackLocator.calcLocation(StackLocator.java:96)
at org.apache.logging.log4j.util.StackLocatorUtil.calcLocation(StackLocatorUtil.java:99)
at org.apache.logging.log4j.core.impl.Log4jLogEvent.getSource(Log4jLogEvent.java:693)
at com.snaplogic.cc.log.JsonLogLayout.writeFileInfo(JsonLogLayout.java:136)
at com.snaplogic.cc.log.JsonLogLayout.writeAllFields(JsonLogLayout.java:109)
at com.snaplogic.cc.log.JsonLogLayout.toSerializable(JsonLogLayout.java:91)
at com.snaplogic.cc.log.JsonLogLayout.toSerializable(JsonLogLayout.java:43)
at org.apache.logging.log4j.core.layout.AbstractStringLayout.toByteArray(AbstractStringLayout.java:282)
at org.apache.logging.log4j.core.layout.AbstractLayout.encode(AbstractLayout.java:209)
at org.apache.logging.log4j.core.layout.AbstractLayout.encode(AbstractLayout.java:37)
at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:197)
at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:190)
at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:181)
at org.apache.logging.log4j.core.appender.RollingRandomAccessFileAppender.append(RollingRandomAccessFileAppender.java:243)
at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:161)
at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:134)
at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:125)
at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:89)
at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:542)
at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:500)
at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:483)
at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)
at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:82)
at org.apache.logging.log4j.core.Logger.log(Logger.java:161)
at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2205)
at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2159)
at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2142)
at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2028)
at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1891)
at org.apache.logging.slf4j.Log4jLogger.debug(Log4jLogger.java:134)
at com.snaplogic.cc.snap.common.SnapManagerImpl.markAsFailed(SnapManagerImpl.java:1088)
at com.snaplogic.cc.snap.common.SnapHolder.lambda$setAdditionalPropertyValues$0(SnapHolder.java:248)
at com.snaplogic.cc.snap.common.SnapHolder$Lambda$622/0x0000000800a10440.accept(Unknown Source)
at com.snaplogic.snap.api.ErrorViewsImpl.write(ErrorViewsImpl.java:268)
at com.snaplogic.snap.api.rest.RestResponseProcessor.handleSnapDataException(RestResponseProcessor.java:847)
at com.snaplogic.snap.api.rest.RestResponseProcessor.process(RestResponseProcessor.java:412)
at com.snaplogic.snap.api.rest.RestRequestExecutor.execute(RestRequestExecutor.java:372)
at com.snaplogic.snap.api.rest.RestRequestExecutor.executeOnce(RestRequestExecutor.java:303)
at com.snaplogic.snap.api.rest.RestRequestExecutor.execute(RestRequestExecutor.java:186)
at com.snaplogic.snap.api.rest.RestRequestExecutor.execute(RestRequestExecutor.java:169)
at com.snaplogic.snaps.rest.Get.process(Get.java:124)
at com.snaplogic.snap.api.ExecutionUtil.process(ExecutionUtil.java:106)
at com.snaplogic.snap.api.ExecutionUtil.executeForDocument(ExecutionUtil.java:118)
at com.snaplogic.snap.api.ExecutionUtil.execute(ExecutionUtil.java:81)
at com.snaplogic.snap.api.SimpleSnap.execute(SimpleSnap.java:70)
at com.snaplogic.cc.snap.common.SnapRunnableImpl.executeSnap(SnapRunnableImpl.java:803)
at com.snaplogic.cc.snap.common.SnapRunnableImpl.execute(SnapRunnableImpl.java:585)
at com.snaplogic.cc.snap.common.SnapRunnableImpl.doRun(SnapRunnableImpl.java:869)
at com.snaplogic.cc.snap.common.SnapRunnableImpl.call(SnapRunnableImpl.java:435)
at com.snaplogic.cc.snap.common.SnapRunnableImpl.call(SnapRunnableImpl.java:117)
at java.base@11.0.8/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base@11.0.8/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base@11.0.8/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base@11.0.8/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base@11.0.8/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base@11.0.8/java.lang.Thread.run(Unknown Source)

Error Fingerprint[0] = efp:java.util.stream.2_50vZVT`

[bojanvelevski]

Hey @Prasa,

Try changing the response entity type to TEXT.

[Prasa]

Changing to response entity type- TEXT did not help ☹️

[Prasa]

Hi Team,
Any solution for this issue?

[alchemiz]

If you can get the total byte size what I can suggest is to do a multi-part request using http request header range