Directory Browser vs File Poller snap
Hi,
I am new to Snaplogic. I would like to read all the files in a specific directory at specific intervals. I am evaluating two options. Would like to know the pros and cons of each.
Option 1: D...
Forum Discussion
What are your intervals? If less than 5 minutes, I recommend the File Poller approach and move files to a “working” directory to ensure the same files aren’t polled for multiple runs. Keep in mind that the file poll will run continuously until the timeout is reached. So even if it finds files to process, it will send those on to the next snap and continue polling. Depending on how long it takes to process your files, it is possible to poll the same files before you finish, so this may be challenging for you to implement if you aren’t familiar with the usage.
If your interval is more than 5 minutes, you can use a Directory Browser in your pipeline. Create the task as scheduled and enable the “Do not start a new execution if one is already active” option to prevent multiple instances. This is a simpler pattern and easier to implement.
This post is a few years old but I wanted to get clarity on this statement. You seemed to indicate that at the end of each polling interval, the output is passed downstream and polling will continue until the time out. That has not been my experience. The output for me was only passed on once the timeout was hit. Items were added to the output at each polling event if they were added to the file location being polled but not until the timeout is reached does the output move on.
Is there a configuration for the snap that will achieve the behavior you experienced?
I have experimented with different combinations of “Only Output on Change” and “Exit on first matches” but have not achieve the behavour as you describe.
Has your experience changed with this File Poller snap?
Thanks,
