Error while executing sequence - NotAcceptableIOException : Not acceptable for testsnap2lsat: { "error_map" : null }
Hi…I’m getting the below error in my sequence on random pipelines. It automatically gets resolved and repeats again, however the child pipelines when executed individually works fine. Does anyone have an idea on this? Any information would be helpful! com.snaplogic.snap.api.SnapDataException: Failed to execute pipeline at com.snaplogic.snaps.flow.PipeExec.executePipeline(PipeExec.java:1121) at com.snaplogic.snaps.flow.PipeExec.process(PipeExec.java:802) at com.snaplogic.snaps.flow.PipeExec.processSafely(PipeExec.java:627) at com.snaplogic.snaps.flow.PipeExec.execute(PipeExec.java:486) at com.snaplogic.cc.snap.common.SnapRunnableImpl.executeSnap(SnapRunnableImpl.java:812) at com.snaplogic.cc.snap.common.SnapRunnableImpl.execute(SnapRunnableImpl.java:586) at com.snaplogic.cc.snap.common.SnapRunnableImpl.doRun(SnapRunnableImpl.java:877) at com.snaplogic.cc.snap.common.SnapRunnableImpl.call(SnapRunnableImpl.java:436) at com.snaplogic.cc.snap.common.SnapRunnableImpl.call(SnapRunnableImpl.java:120) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.snaplogic.api.ExecutionException: Prepare on neighbor failed at com.snaplogic.cc.pipeline.PipelineManager.plexExecute(PipelineManager.java:2395) at com.snaplogic.cc.pipeline.PipelineManager.execute(PipelineManager.java:2448) at com.snaplogic.snapi.pipeline.builder.PipelineExecutionBuilderImpl.execute(PipelineExecutionBuilderImpl.java:220) at com.snaplogic.snapi.pipeline.builder.PipelineExecutionBuilderImpl.retry(PipelineExecutionBuilderImpl.java:185) at com.snaplogic.snaps.flow.PipeExec.executePipeline(PipeExec.java:1021) ... 14 more Caused by: com.snaplogic.util.NotAcceptableIOException: Not acceptable for testsnap2lsat: { "error_map" : null } at com.snaplogic.util.HttpUtils.getInputStreamFromResponse(HttpUtils.java:250) at com.snaplogic.util.HttpUtils.getInputStreamFromResponse(HttpUtils.java:164) at com.snaplogic.cc.pipeline.PipelineManager.plexExecute(PipelineManager.java:2334) ... 18 more Reason: Not acceptable for testsnap2lsat: { "error_map" : null } Error Fingerprint[0] = efp:com.snaplogic.snaps.flow.VugOJsmW Error Fingerprint[1] = efp:com.snaplogic.cc.pipeline.fqgzTyoF Error Fingerprint[2] = efp:com.snaplogic.util.aWBrAYmz
Ingest large number of small files as a sequence file and store file metadata relating to each file
Created by @pkona This pipeline pattern ingests a large number of small files as a sequence file and stores file metadata relating to each file. If you have a number of small files on SPTP/S3/HDFS/Kafka, you may want to store these files and their metadata into a larger file for efficient storage. You can use this pattern if you have multiple small files each relating to a certain event, such as a few hundred of small files generated in an hour. This pattern retrieves the files, extracts the metadata and ingests as a single sequence file that contains zipped source files and file metadata. The files can then be written to any target location using the File Writer Snap including S3, WASB, ADL, HDFS, SFTP, and other protocols. Configuration Sources: file Targets: sequence file Snaps used: Directory Browser, File Reader, Compress, Binary to Document, Mapper, Sequence Formatter, HDFS Writer Downloads IngestPattern - Ingest multiple small files as SequenceFile.slp (11.4 KB)
