cancel
Showing results for 
Search instead for 
Did you mean: 

How can we pass a dynamically assigned variable from one end of a pipeline to the other?

swright
New Contributor III

Our organization has decided that they only want a single pipeline for each integration we develop (no execute pipeline to pass parameters to another pipeline). They say that this is due to the way their error handling works and that they only want one pipeline referenced when they have an error.

I explain the above because people will probably suggest assigning the variable in a parent pipeline and passing it as a parameter to a child.

I need to pass variables assigned earlier in a pipeline to snaps that occur later in the same pipeline. For instance, I assign a filename variable like:

“test_”+Date.now().toLocaleDateTimeString({“format”:“yyyyMMdd”})+“.csv”

I use it in a File reader snap at the beginning of the pipeline and need to use it again multiple times near the end of the pipeline. At the end of the pipeline I move the file to an archive location and need the file name to do that.

I can’t set parameters dynamically in “Edit Pipeline” or within the pipeline, and it is difficult to pass a variable from one end of the pipeline to the other. In searching this forum I find that others have this same issue but I didn’t see any solutions other than using parent/child pipelines. Is there an easy way to set a variable early in a pipeline and use it in other snaps that come later in the same pipeline? Other workarounds…?

Thanks!

12 REPLIES 12

wpenfold
Contributor

2020-11-02 15_04_23-snaplogic_notes.docx - Word

You probably need to specify passthru on most snaps

swright
New Contributor III

Using passthru results in some messy json that can complicate things by the time it gets to the end, and it seems that there are some snaps that don’t have the passthru option.

I’m not sure how I can use jsonpath($.“$…PARAMNAME”) for this issue. Can I pass the variable somehow using this?

Thanks.

You define PARAMNAME at the beginning of the pipeline, use passthru where possible, and when you want to use the PARAMNAME later, you use jsonpath to get it.

Other possibility is you get the PARAMNAME value, then use a COPY snap with multiple outputs. Then you use a JOIN snap to bring the value back into the pipeline at other later locations where you need it.

swright
New Contributor III

Thanks for the explanation! This sounds like a useful techique.