Match Expression stored in a Variable
I have a Pipeline using in different branches the same Mapper for one of the Field
See ScreenShot
I would like to use a Pipeline Variable in my Mapping Expression in order to update/maintain...
Forum Discussion
Max - The simplest way that I can think to accomplish what you're asking for is to use a Gate snap with the default settings and rename the input view to "exchangeRates" as below:
A word of caution on the Gate snap with the default settings - this snap consumes all input into a single document, which may consume significant memory resources if processing a large volume of data.
Hope this helps!
koryknick- The gate snap worked in this use case. However, it is concerning that it may have resource issues if processing large data. The data set in this case is around 35K records. But we have a lot of cases this logic will need to be used and our datasets are huge. Is there a better way/more efficient way of doing this with big result sets?
Hello,
My suggestion rather than using tail snap use Group By N snap and keep the Group Size as 0. It will wait for all the documents which are processing in first pipeline and then you can map your object after Group By N something like $group.Date etc… and call the second process.I also use the Tail Snap to control the flow of the execution. Please share the setting of the second Pipeline Execute Snap. Do you have the “reuse” on? If so, the pipeline will start as soon as the main pipeline starts, but if the “reuse” is off, it should start when there is an input document getting into the Snap.
