Forum Discussion

Siva_Venna's avatar
Siva_Venna
Contributor
5 years ago
Solved

Snaps execution in pipeline

I ran into a typical scenario where snaps are not executing in the order they are supposed,below is the screenshot shows the order snaps are being executed. need help me to resolve this.Thanks full_incremntl_sql_2020_09_28.slp (13.7 KB)

  • Siva_Venna's avatar
    Siva_Venna
    5 years ago

    @lazo.ilijoski
    Probably order of execution of the unconnected snaps/streams depends on the order of creation of the starting snaps in the pipeline - I think execution is not based on order of creation if so my pipeline would have executed as expected. Anyways it’s good to have snaps connected to avoid this kind of issues. will consider your suggestion.Thank you

5 Replies

  • lazo_ilijoski's avatar
    lazo_ilijoski
    New Contributor III

    Hi @Siva_Venna,

    The snaps Oracle-Insert and Oracle-Select1 are not connected (connector is white, instead of blue), and that’s why the order of the execution of the snaps is not as expectd.

    Regards,
    Lazo

    • Siva_Venna's avatar
      Siva_Venna
      Contributor

      Hi @lazo.ilijoski
      in my pipeline the snap left unconnected intentionally, prior to this pipeline I have designed another pipeline as you see below, in this also I have an unconnected snap but it is working as expected for me!

  • lazo_ilijoski's avatar
    lazo_ilijoski
    New Contributor III

    Hi @Siva_Venna,

    Probably order of execution of the unconnected snaps/streams depends on the order of creation of the starting snaps in the pipeline. Since, the order of the executions cannot be guaranteed and if you want to guarantee the order of executions, then I suggest all snaps to be connected by using copy/router or any other appropriate snap.

    Regards,
    Lazo

    • Siva_Venna's avatar
      Siva_Venna
      Contributor

      @lazo.ilijoski
      Probably order of execution of the unconnected snaps/streams depends on the order of creation of the starting snaps in the pipeline - I think execution is not based on order of creation if so my pipeline would have executed as expected. Anyways it’s good to have snaps connected to avoid this kind of issues. will consider your suggestion.Thank you

  • All snaps in a pipeline are initialized in parallel. After initialization, each snap waits for input from its upstream snap. As data becomes available, the data is processed by the snap and written to the downstream snap. The control flow is driven by the availability of data to process. Snaps connected to each other will process a document in the same order as the order they are connected to each other in.

    If there are multiple unconnected branches in the pipeline, each of them processes data in parallel. There is no synchronization between pipeline branches. Disconnecting two snaps which are next to each other functionally makes them unconnected branches.