Effect of Pipeline failure

When a pipeline fails, there are some snaps whose execution stops, want to understand how does it actually happen?

Example, I have the below pipeline,

when one of the pipeline in this fails I see the below statistics,

I do not understand why the other independent pipeline’s execution should be affected?

The pipeline execution can be stopped in two particular ways.

  1. If the underlying configuration of any snaps are not correct. In this
    case, the pipeline execution will stop the current execution and the snap
    will error out.
  2. If the underlying snap has errors view enabled, then the error view
    will capture the underlying errors.

It will error out in the above order.

In your particular case, the pipeline execute failed because of underlying
child pipeline errors. If you had errors view enabled, you would have seen
it there.

If your child pipeline had 2 unconnected outputs then the snap would have
failed right during the execution while preparing for it.

Hope that makes sense.

Thanks & Regards,
Naveen

Thank you for the information, however I am looking to understand why should failure of one child pipeline affect / stop the other child pipeline.

In my case there 2 child pipelines I am calling using pipeline execute. Now when I have an error in child1 its failure is obvious, but why is the execution of child2 be stopped?

Did you have the error view on the pipeline execute and experience this
issue? Can you enable the error view and let us know how it behaves?

Thanks & Regards,
Naveen

To clarify, child executions are not independent of their parent. All child executions are governed by their parent. If the parent is stopped, all of its child executions will be stopped as well. This behavior was chosen so that it was not possible to lose control and have runaway executions.

When child1 fails, the PipeExec snap will signal the failure by writing an error document. Since the PipeExec snap does not have the error view open and is configured to stop pipeline execution in case of an error, the snap will be marked as failed and the parent pipeline will be stopped. And, since the parent is stopped, all of the children will be stopped as well.

Ok, thanks you for the explanation. I have enabled the error view on the pipeline execute snap and I could achieve what I intended to.

Hi Team,

I am also facing the similar issue. I am using Pipeline Execute to invoke the child pipeline execution.

Child pipeline has Error pipeline defined. My observation is when there is an error in child pipeline it invokes the error pipeline and stops the execution of Parent pipeline.

E.g. Pipeline execute is supposed to call child pipeline to process 19 JSON files. If there is an error in 17th JSON file, execution of Parent pipeline also stops and it won’t process 18th and 19th file.

image

Can you please suggest?

Regards,
Ajit Bodkhe.

You could try the following:

Add the Error View to the pipexec snap so that any error that comes from the child pipeline, it will log the error and continue. This way the parent pipeline will not terminate.