Forum Discussion

shreegeet's avatar
shreegeet
New Contributor II
6 years ago

Error Handling while Connecting SFTP

Hi,

I have a requirement wherein I have to Pull all the files from a particular folder of SFTP and then Process them. I have a couple of questions on error handling part of it

  1. How should I check that there is no error while fetching the files?
  2. What if only a few files are pulled by snap and then the connection is lost? In this scenario, only some files will be processed and there would be data inconsistency

Any suggestions to handle this scenarios

Regards,
Shreegeet

1 Reply

  • alchemiz's avatar
    alchemiz
    Contributor III
    6 years ago

    So, when you said “Pull” will you be reading then processing or would you download the SFTP files into a temp folder (pipe.tmpDir) then read from that temp folder then process the file/s?

    For a successful/complete downloading of files, I would suggest to enable the “Validate after write” in the File writer snap