Scheduled Task + File Poller guidelines
I have several legacy file-based interfaces I’m migrating to Snaplogic. Many of them involve polling for new input files on a scheduled basis, usually something like every 30/60/120 seconds. I’ve c...
Forum Discussion
@marenas - Unless you are running the File Poller snap in an Ultra Task which will automatically restart the task as soon as it closes, I would recommend that you schedule the task for the minimum acceptable “downtime” in the event the File Poller terminates abnormally. As of the 4.26 release (Sep '21), SnapLogic enabled the Snaplex-based Scheduler, which means that scheduled task timing and reliability is dependent on your local snaplex and should be very close to the selected timing.
With that stated, you could set your File Poller to check for file existence as often as you wish, with the timeout set for 59 minutes (for example), then create the Scheduled Task to execute every 5 minutes with the “Do not start a new execution if one is already active” option enabled. This means that the polling would only be “down” at most 5 minutes every hour. Since the scheduler is using local snaplex resources, you could even have the task scheduled every minute if you desire.
@koryknick thank you for your response.
In my sample pipeline I set up a file poller with these properties below:
A scheduled task is set to run daily at 12 midnight (“Do not start a new execution if one is already active” option enabled). I expected that the behavior of this pipeline will continue to poll the matched file and output only when there is a change in the contents of the polled directory. Is that correct? The file has less than 100 rows but should I increase the polling interval to make sure enough time is allocated to process each file?
