Error Handling while Connecting SFTP

[shreegeet]

Hi,

I have a requirement wherein I have to Pull all the files from a particular folder of SFTP and then Process them. I have a couple of questions on error handling part of it

1. How should I check that there is no error while fetching the files?
2. What if only a few files are pulled by snap and then the connection is lost? In this scenario, only some files will be processed and there would be data inconsistency

Any suggestions to handle this scenarios

Regards,
Shreegeet

[alchemiz]

So, when you said “Pull” will you be reading then processing or would you download the SFTP files into a temp folder (pipe.tmpDir) then read from that temp folder then process the file/s?

For a successful/complete downloading of files, I would suggest to enable the “Validate after write” in the File writer snap