cancel
Showing results for 
Search instead for 
Did you mean: 

Get latest file from sftp and process one at a time

hina_walia
New Contributor

I want to get all files from yesterday’s date from ftp and process them one by one. There can be only one file and can be multiple files but they need to be processed one at a time. I tried using mapper , to get yesterday value “Date.now().toLocaleDateString(‘{“format”:“yyyyMMdd”}’)”
and then use file poller, and write in file filter Labor_FDC_${ye}*.dat,
but it doesn’t work. Also at the end after load, i want to update file name in target table that it is processed ,so that it is not processed again.

17 REPLIES 17

If i hardcode value in child pipeline it works, but variable doesn’t work. Please see attached screenshot, first is
child pipeline set parameter and second is reading variable from file reader in child pipeline . Can you please check if syntax is fine. Thanks in advance.
child_pipeline_parameter_set child_pipeline_with_path_variable

dmiller
Admin Admin
Admin

Pipeline parameters only take string values, not field references.
To test this child pipeline by itself, set the Value in the Edit Pipeline dialog to the path value like sftp://xxx.xx.x.xx/abc.dat


Diane Miller
Community Manager

Thanks . working fine now.