Pipeline stop itself without failing
Is there a way for a pipeline to stop itself without ending in the Failed state as the Exit snap does?
Forum Discussion
What are you looking for as the final state of the execution? Do you just want it to be STOPPED instead of FAILED? Or, do you want to be marked as a successful completion? If the former. What’s the significance of changing from failed to stopped for you? If the latter, can you describe the structure of the pipeline and why processing cannot be completed cleanly? We might be able to identify a structure that would not require the exit snap.
Since the snaps in a pipeline are all running at the same time and there can be documents in-flight, stopping a pipeline is a pretty harsh operation. So, there isn’t much difference between STOPPED and FAILED, other than being able to identify that the STOPPED might be expected versus the FAILED is unexpected.
Yes, I agree stopping is harsh and I ultimately would like to ensure there are no in-flight docs. We have JMS Consumer snaps configured to keep pipelines continually running. I have the following use cases where I want to report the snap as Completed, not Failed, as these are controlled stops.
Use case 1: I need the pipeline to perform a controlled stop due to a known error: e.g. database connections timeout and there is no retry on our update and insert snaps so I need to restart the pipeline to get new connections.
Use case 2: (If the Control Destination just pauses the JMS Consumer snap and doesn’t stop the JMS Consumer snap, which results in stopping the pipeline, then) I need a controlled stop after receiving some control message to be able to stop the pipeline softly.(I just discovered the Control Destination on this snap and I need to research how to use them to stop listening to ensure no docs are in-flight.)
