โ08-11-2021 02:30 PM
Hi,
I am struggling with what seems to be a simple pipeline.
Here are my requirements:
1.Get list of file names from the database (original file name and new file name)
2. Read each file from one SFTP server using โoriginal file nameโ and then write it to a different SFTP server renaming it to the โnew file nameโ.
My issue is that I am unable to pass โnew file nameโ after I use file read snap.
File read snap does not have a โPass Throughโ option.
How can I do that?
Thank you
โ08-11-2021 10:24 PM
Hi,
Create a master pipeline and get list of file names from the database (original file name and new file name). After collecting file names, place a pipeline execute snap and invoke the child pipeline. In the pipeline execute snap,configure the filenames as below.
In the child pipleine, place a file reader snap provide the details to read the original file name using _original_file_name pipeline parameter. In the file writer snap, provide the details to create a file using _new_file_name pipeline parameter.
โ08-12-2021 05:22 AM
Hi,
This looks like a work around the problem.
Weโll have over 10000 files to rename/move every time.
Is that going to be an issue to execute child pipeline that many times?
Thanks.