Thanks @cjhoward18. I will investigate more about the desired behavior and apply your suggestion if that’s what’s needed. I think in this case the error pipeline itself was misconfigured and hanging. Thus, the calling pipeline was still showing running (or successful completion), but if you looked into the statistics there were errors. Once the configuration is fixed it should either succeed or fail, and the hanging should no longer be a problem. Thanks again for your quick response.
I don’t think this is documented in the error pipeline documentation, but this is actually the expected behavior. Say you’re processing 500 documents and the 100th document in is munged and will result in an error when processing. A user may still want to execute on the other 499 valid documents and record which document is causing an issue. Additionally teams generally want to standardize on the error handling, so error pipelines allow all pipelines to process errors in a standard way.
When viewing a pipeline configured with an error pipeline, you can view the errors by looking at the pipeline execution stats. These show how many documents pass through the output and error view respectively.
As @cjhoward18 mentioned, you can add an Exit snap to the error pipeline if you want the parent pipeline to fail and stop processing. Here’s an example of that if you want to try it out.
projects_error-pipeline-example.zip (2.5 KB)
You can also modify the error handling behavior of each snap individually.
Hope this helps clarify and give you some starting points when building out the error handling in your integrations.
Best
