โ08-31-2021 01:36 AM
Hi All,
I need to give a s3 path dynamically from redshift metadata table in S3 File writer.
Because it has csv formatter before it, not able to give redshift execute and mapper to pass the value.
Any suggestion.
Regards,
Shrabanti
โ08-31-2021 03:23 AM
Hi Shrabanti,
You can create a child pipeline that will contain at least the CSV Formater and the S3 File writer snaps. From there you can send the path value in Pipeline Parameters and you can read the path from the parameter dynamically in the created child pipeline.
Please let me know if you have any questions.
Regards,
Nikola Minovski
โ09-01-2021 12:40 AM
I have passed the value from parent.I can see the value in json
I have declared pipeline parameter in child pipeline
3.But in s3 file reader I am not getting the value I am passing.
Is it something wrong I am doing here?
Thanks & Regards,
SHrabanti
โ09-01-2021 05:04 AM
Hi Shrabanti, I would use the dropdown on the right and click the button on the upper right in the dropdown to visually see if the parameter(s) isnโt showing values.
If you still canโt see the value, checking the parent execute snap and child parameter to verify that there arenโt any typos.
โ09-01-2021 05:30 AM
@Shrabanti As per your 1 step, only directory you have mention, should have filename as well.(because in your file write you didnโt add any filename)
In your step 2, donโt use $ac_feed_path (even you didnโt change technically shouldnโt issue)
Before step 3, use Pipeline execute snap and pass ac_feed_path value from pipeline input param.
Then it should work.
You can pass ac_feed_path to execution label so can track which file has been processed in Dashboard.