Forum Discussion

krupalibshah's avatar
krupalibshah
Contributor
9 years ago

Effect of Pipeline failure

When a pipeline fails, there are some snaps whose execution stops, want to understand how does it actually happen?

Example, I have the below pipeline,

when one of the pipeline in this fails I see the below statistics,

I do not understand why the other independent pipeline’s execution should be affected?

7 Replies

  • The pipeline execution can be stopped in two particular ways.

    1. If the underlying configuration of any snaps are not correct. In this
      case, the pipeline execution will stop the current execution and the snap
      will error out.
    2. If the underlying snap has errors view enabled, then the error view
      will capture the underlying errors.

    It will error out in the above order.

    In your particular case, the pipeline execute failed because of underlying
    child pipeline errors. If you had errors view enabled, you would have seen
    it there.

    If your child pipeline had 2 unconnected outputs then the snap would have
    failed right during the execution while preparing for it.

    Hope that makes sense.

    Thanks & Regards,
    Naveen

    • krupalibshah's avatar
      krupalibshah
      Contributor

      Thank you for the information, however I am looking to understand why should failure of one child pipeline affect / stop the other child pipeline.

      In my case there 2 child pipelines I am calling using pipeline execute. Now when I have an error in child1 its failure is obvious, but why is the execution of child2 be stopped?

      • nganapathiraju's avatar
        nganapathiraju
        Former Employee

        Did you have the error view on the pipeline execute and experience this
        issue? Can you enable the error view and let us know how it behaves?

        Thanks & Regards,
        Naveen

  • ajit_bodkhe's avatar
    ajit_bodkhe
    New Contributor

    Hi Team,

    I am also facing the similar issue. I am using Pipeline Execute to invoke the child pipeline execution.

    Child pipeline has Error pipeline defined. My observation is when there is an error in child pipeline it invokes the error pipeline and stops the execution of Parent pipeline.

    E.g. Pipeline execute is supposed to call child pipeline to process 19 JSON files. If there is an error in 17th JSON file, execution of Parent pipeline also stops and it won’t process 18th and 19th file.

    Can you please suggest?

    Regards,
    Ajit Bodkhe.

    • psathyanarayan's avatar
      psathyanarayan
      Employee

      You could try the following:

      Add the Error View to the pipexec snap so that any error that comes from the child pipeline, it will log the error and continue. This way the parent pipeline will not terminate.