When designing Pipelines we’re often faced with wanting to control the sequencing of Snap execution in different branches – e.g. ‘don’t start executing Snap Y until Snap X has completed’. Often this is solved by using a Union Snap and then a Sort Snap (which collates all documents in the stream). For example:
This is required in order to prevent “Update Task in Env” from starting execution until “Update Pipeline in Env” is complete. Often I also end up doing a Copy of the document stream, Union and then a Filter Snap to exclude the extra copies of the documents.
We’ve requested the feature to have Snaps which allows for flow control / sequencing / dependencies between different branches of a Pipeline, and would allow for simpler and more readable Pipeline designs (Service Request #18184).
I’d be keen to hear any thoughts or feedback from the community - is this a feature you would like to see too?