cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

How can we pass a dynamically assigned variable from one end of a pipeline to the other?

swright
New Contributor III

Our organization has decided that they only want a single pipeline for each integration we develop (no execute pipeline to pass parameters to another pipeline). They say that this is due to the way their error handling works and that they only want one pipeline referenced when they have an error.

I explain the above because people will probably suggest assigning the variable in a parent pipeline and passing it as a parameter to a child.

I need to pass variables assigned earlier in a pipeline to snaps that occur later in the same pipeline. For instance, I assign a filename variable like:

โ€œtest_โ€+Date.now().toLocaleDateTimeString({โ€œformatโ€:โ€œyyyyMMddโ€})+โ€œ.csvโ€

I use it in a File reader snap at the beginning of the pipeline and need to use it again multiple times near the end of the pipeline. At the end of the pipeline I move the file to an archive location and need the file name to do that.

I canโ€™t set parameters dynamically in โ€œEdit Pipelineโ€ or within the pipeline, and it is difficult to pass a variable from one end of the pipeline to the other. In searching this forum I find that others have this same issue but I didnโ€™t see any solutions other than using parent/child pipelines. Is there an easy way to set a variable early in a pipeline and use it in other snaps that come later in the same pipeline? Other workaroundsโ€ฆ?

Thanks!

12 REPLIES 12

wpenfold
New Contributor III

2020-11-02 15_04_23-snaplogic_notes.docx - Word

You probably need to specify passthru on most snaps

swright
New Contributor III

Using passthru results in some messy json that can complicate things by the time it gets to the end, and it seems that there are some snaps that donโ€™t have the passthru option.

Iโ€™m not sure how I can use jsonpath($.โ€œ$โ€ฆPARAMNAMEโ€) for this issue. Can I pass the variable somehow using this?

Thanks.

wpenfold
New Contributor III

You define PARAMNAME at the beginning of the pipeline, use passthru where possible, and when you want to use the PARAMNAME later, you use jsonpath to get it.

Other possibility is you get the PARAMNAME value, then use a COPY snap with multiple outputs. Then you use a JOIN snap to bring the value back into the pipeline at other later locations where you need it.

swright
New Contributor III

Thanks for the explanation! This sounds like a useful techique.