I have several legacy file-based interfaces I’m migrating to Snaplogic. Many of them involve polling for new input files on a scheduled basis, usually something like every 30/60/120 seconds. I’ve coded the ones I’ve done using a combination of a scheduled task that calls a pipeline containing a file poller. I’ve played around with having all of the iteration done by the task and having the file poller only run once, and I’ve also tried setting the task to run every 30/60/90 minutes and having the file poller run every 30/60/120 seconds with a timeout equal to the task frequency. I like doing the latter since I can iterate at a higher frequency, but given that task scheduling is somewhat approximate, getting the the two schedules in sync is more of a problem.
Is there any general guidance as to how to set up the two schedules, i.e. is it better to schedule a task to run once an hour/day/week and then have the poller do all of the smaller iterations in that larger time-slice? If so, what should the task frequency generally be?
Thanks in advance.