Updating Pipeline Parameters

[stephenknilans]

How can we change Pipeline parameters through snaplogic(programmatically) , or the environment instance?

[Harriesh]

To my knowledge only way is to create a trigger task for invoking the pipeline and Call it with required Pipeline parameters

[stephenknilans]

I ended up doing that. Still, it is a shame that there is no may to provide global variables within a process.

[tstack]

stephenknilans:

How can we change Pipeline parameters through snaplogic(programmatically) , or the environment instance?

The problem with changing pipeline parameters during an execution is that it makes it really easy to introduce unpredictable results. For example, if Snap A references a parameter that is changed by Snap B, the value of the parameter read by A would be completely unpredictable since all of the Snaps are running in parallel. In other words, the value of the parameter read by Snap A might be “foo” for the first 20 docs in one run and then “bar” after that. But, if the pipeline was run again with the same inputs, the parameter might be “foo” for the first 10 docs instead of the first 20. So, to avoid these race conditions, parameters are treated as static.

If you need to change a parameter for a subsection of a pipeline, you can move that subsection into a child pipeline and call it with Pipeline Execute, setting the parameters as needed.

Harriesh:

To my knowledge only way is to create a trigger task for invoking the pipeline and Call it with required Pipeline parameters

Are you calling the task from another pipeline in this case? Is there a reason the Pipeline Execute snap does not work for you?

[stephenknilans]

Tstack,
Admittedly I didn’t think about the aspect you just explained. Still, it would be nice to change them at the beginning. I’ll be happy with what I have at the moment. That uses a filereader to read a normal csv file, uses a script to pivot it, and a mapper to potentially change values that are then used to execute a pipeline that runs everything according to those variables, passed as parameters. So everything runs appropriately in the proper environments. It is nice to tell the customer Export the pipeline, import into the new environment, and simply RUN! Of course we give them docs, and a little course so they can provide their own support, but initially it is made to be SIMPLE.

Of course there IS that design flaw in the reltio write snap. It doesn’t allow ANY programability. If we could merely use a pipeline parameter to provide the tenant URL, we would be SET! You guys indicated it would be fixed in the new release. It looks like that is coming out TOMORROW. Does that mean WE can use it in QA? It would be great if we could. We just had our first go live without this, and I want this to work well from the beginning.

If we weren’t going to get access, can you make it so we will?

Steve