Get latest file from sftp and process one at a time

[hina_walia]

I want to get all files from yesterday’s date from ftp and process them one by one. There can be only one file and can be multiple files but they need to be processed one at a time. I tried using mapper , to get yesterday value “Date.now().toLocaleDateString(‘{“format”:“yyyyMMdd”}’)”
and then use file poller, and write in file filter Labor_FDC_${ye}*.dat,
but it doesn’t work. Also at the end after load, i want to update file name in target table that it is processed ,so that it is not processed again.

[dmiller]

Provided that the account configured on the File Reader has access to the desired path location, it might be easier if someone looked at your pipeline. If you can’t post it here, you might want to have your Customer Success Manager (CSM) or Support look at it. I have alerted your CSM in case.

[hina_walia]

Hi
when i check in pipeline preview, i can see the value, but in child pipeline it looks like it is not getting passed through.

[hina_walia]

please see attached error in child pipeline

Below is screenshot parent pipeline execute shows the param preview value correctly but it is not passed in child pipeline

[dmiller]

hina_walia:

when i check in pipeline preview, i can see the value, but in child pipeline it looks like it is not getting passed through.

The child pipeline itself does not pull information from parent pipelines; parent pipelines send information to them. This allows you to reuse child pipelines with other pipelines. To test a child pipeline by itself, enter a value for your pipeline parameters.

[hina_walia]

is it possible to send me example. When i tried to hardcode value in child pipeline, it works fine but when try to get variable from parent pipeline, it throws error.Thanks a lot