Directory Browser vs File Poller snap

[mramaswamy]

Hi,
I am new to Snaplogic. I would like to read all the files in a specific directory at specific intervals. I am evaluating two options. Would like to know the pros and cons of each.
Option 1: Directory Browser snap with Task
Option 2: File Poller snap

Also I would like to make sure that the processing of the pipeline is completed before the next polling/browsing of files.

[koryknick]

Thanks @skatpally - I should have double-checked the documentation on the File Poller snap. It’s been a while since I’ve used it.

@mramaswamy - back to your original question. I think the Multi-File Reader snap is probably your simplest option to do what it sounds like you are doing.

The File Poller is typically used in an “always on” situation to poll continuously during a pipeline execution, for example in an Ultra Pipeline where it might poll the directory many times a minute, 24 hours a day.

[mramaswamy]

Thanks for all your help. This is really helpful. I have not tried multi-file reader. At this point we are comfortable with Directory Browser + File Reader + Task scheduler. I will keep posted after we try multi-file reader.

[koryknick]

If you are trying to view the behavior in Pipeline Validation, it gets a bit confusing because you don’t see the data preview until it either times out or outputs enough documents to satisfy your Document Preview Count in your User Settings.

You should be able to see the correct behavior in the Execution Statistics in Dashboard… i.e. each time it picks up a file, the output document count on the File Poller snap will increase.