cancel
Showing results for 
Search instead for 
Did you mean: 

Reusable pipeline- Unable to fetch variable value in child pipeline -Snowflake Snap

smahla
New Contributor II

Usecase-
We are looking at creating a reusable pipeline to load data from table1 from source DB to table1 in target DB. Plan is to reuse the same pipeline for n tables.

Option 1 -
Created master pipeline and child pipeline. Master pipeline gets table name list from source DB.
Then uses mapper and foreach snap to execute the child pipeline.
Child pipeline loads table from source DB to target DB.
Issue- Unable to use the parameter/variable value from master pipeline-foreach snap to child pipeline -snowflake select/execute snap.

Master pipeline-
image

Child pipeline-
image

Anyone faced similar error or has recommendation for the approach on reusable pipeline.

Thank You.

1 ACCEPTED SOLUTION

Hi @smahla,

You are getting that error while validating from the child pipeline because the Parameter _name is empty.
image

You can specify Default Values for the Parameters so you can validate, and these values will be overwritten when you invoke the Pipeline from the parent.

So, if you want to test it by validating directly the child pipeline consider setting the Default Value for the wanted Parameter under the Pipeline Properties.

Let me know if this helps. 🙂

BR,
Aleksandar.

View solution in original post

7 REPLIES 7

Hi @smahla,

You are getting that error while validating from the child pipeline because the Parameter _name is empty.
image

You can specify Default Values for the Parameters so you can validate, and these values will be overwritten when you invoke the Pipeline from the parent.

So, if you want to test it by validating directly the child pipeline consider setting the Default Value for the wanted Parameter under the Pipeline Properties.

Let me know if this helps. 🙂

BR,
Aleksandar.

jcornelius
New Contributor III

You need to set _name if you are going to use “Pipeline Parameter”
You can use “Document Parameters” and they are referenced with $. Lokk at the output of Snowflake - Execute by clicking the Red doc icon, this will show into the next snap.
ie is output is “tableName”: “table1” then change select to "select * from " $tableName

smahla
New Contributor II

Thanks @AleksandarAngelevski , @jcornelius , @bojanvelevski
Not sure on how to use document parameters. Tried with pipeline parameters and assigned default values for pipeline parameter to get rid of validation error in child pipeline. During execution, child pipeline parameter default values got over written by values passed by parent pipeline.

Thank you.