Parent/Child Parameters

[cew657]

I’m new to Snaplogic. I am working on a solution to pull files from a network share and load to a Snowflake table. I have a Parent set up to pull the file names from the network share using the directory browser snap which is going to the Pipeline Execute which then runs the Child pipeline. I am including the full path and filename in my Parameter.

My issue is how to pass the parameters from the Parent to the Child. I have downloaded several examples of this from the boards and I still can’t get it to work.

I can see the file names in the preview of the Pipeline Execute snap of the Parent pipeline. When I switch to the Child pipeline, the parameter is whatever is defined in the Pipeline Properties box when I do a preview.

How do I pass the Parent parameters to the Child Parameters? The file writer is using smb://network drive location/shared/xxxxx/xxxx/file name which is why the equal sign is not depressed in the File Reader of the child snap as for some reason smb does not work with the equal sign depressed.

Thanks for any assistance.

Parent:

Parent Pipeline Execute:

Parent Pipeline Properties

Child:

Child Pipeline Properties:

Mapper Snap from Child:

Mapper Snap Child Preview

[alex_panganiban]

In the snap, Mapper1, that’s in your parent pipeline (the snap right before the Pipeline Execute snap), are you mapping your directory browser results to $Filename? You didn’t show a screenshot of what’s inside that snap, but everything else looks in place. As long as Mapper1 is passing a value to $Filename, you should be okay.

[cew657]

This is what is in Mapper 1 of the Parent. Since I can see the file names in the Directory Browser, Mapper 1, and Pipeline Execute snaps of the Parent, I would think they would carry over to the Child.

When I view the output of Mapper2 in the child there are no values. Should I see the values I am seeing from the Parent which should be the list of files that I can pass down the stream?

How would I use the Parameters in the Child? In the File Reader of the Child, I’ve tried both _FileName and $FileName and keep getting an error and I think it has something to do with the Mapper 2 values not coming in from the Parent.

Mapper 1 from Parent

File Reader from Child

[bojanvelevski]

Hi @cew657,

You’re actually pretty close. The parameters you send while executing/validating will override the ones you’ve set in the child pipeline. You cannot see those directly, but you can capture them like you already did in your setup. Those can be seen on the execution within the Dashboard.

Next, the only problem here I think is that you are passing the filename as a parameter, not the file path, which I believe it is produced from the Directory Browser as a $Path field.

After we confirm that this is the issue you’re having, I think you can try and simplify things significantly, because you don’t need to pass any parameters at all. After the Directory Browser lists the files and paths, the data will enter the child pipeline, and the $Path field can be directly passed into the File Reader, no need for parameters, no need for mapping.

Try these and let us know the result.

Regards,
Bojan

[cew657]

Thank you everybody for the input/assistance. I made the changes to the Parent pipeline to use the $Path. I was not sure how to pass the $Path field directly to the child without Parameters or run the Child without the Pipe Execute snap. With this process I am setting up, I can have anywhere from 1 to 15 files a day go into the directory.

I then modified the Child pipeline by removing the Mapper and then using the Parameter of _FileName in the reader.

With this new configuration, I am finally able to get a successful validation. However, the File Reader snap is always red on validation. Is this normal? In the Child Pipeline screen shot you can see this is Red.

I am able to run the pipelines and the data is being loaded successful based on record counts in the target table.

Parent Pipeline with Changes

Child Pipeline

Child Pipeline with Parameter