Using a File Poller to a File Reader

#1

Hello,

I am trying to figure out how to use a File Poller to make a file name a variable and then to use that variable as the file to use in a File Reader? I am not sure on how to get it to work. Also, how does the File Poller truly work if I have multiple files with the same name, but different timestamps, does it go through them 1 by 1? or all 3 at the same time for example? I using a file that is on an FTP server, and is dynamic with a time stamp, and there could be multiple files but with different timestamps.

#2

Yes It does keep polling one by one. Poller Snap polls the target directory and looks for file names matching the specified pattern. It keeps on polling at the Polling Interval property until the polling timeout is reached.The Snap returns all paths that matches the File Filter Pattern.

o/p

“path”: “ftp://XX.XX.XX.XX/HousingPhotos1_FullFeed.zip”
“path”: “ftp://XX.XX.XX.XX/HousingPhotos1_FullFeed.zip”
“path”: “ftp://XX.XX.XX.XX/HousingPhotos1_FullFeed.zip”

You can do a Split on the path and pass that to the File Reader Snap.

Split Function

$path.split(’/’)[$path.split(’/’).length-1].toString()

09%20PM
Screen Shot 2018-11-26 at 12.18.09 PM.png711×459 21.3 KB

Hope this helps.

#3

What would the file reader use to read the file from the Poller. Because my pipeline keeps having errors cause I don’t know what the file name/variable needs to be to read it in the File Reader

#4

You can directly pass the $path to the File Reader snap and read the data. Make Sure you have the Account selected in the Account Tab. Please post the error and sample pipeline for reference. Thank you.

#5

That is my pipeline. I am not sure if I am doing this correctly, I have the variable name set up in the Poller, and In the File read I have the account set up, but still don’t understand the dynamics of the file name. Do I need to change the output in views to $path on the Poller Snap.

pl_street_level_fs_trade_2018_11_26 (1).slp (40.0 KB)

#6

If you validate the Pipeline - File Poller Snap should return the o/p. Something like this

o/p

“path”: “ftp://XX.XX.XX.XX/HousingPhotos1_FullFeed.zip”
“path”: “ftp://XX.XX.XX.XX/HousingPhotos1_FullFeed.zip”
“path”: “ftp://XX.XX.XX.XX/HousingPhotos1_FullFeed.zip”

The in the File Reader Snap - File* Just pass $path.

#7

Hi Falleur,

I would suggest use directory browser over file poller if integration type is scheduled. Once you read file from directory browser you can found the path variable as a output , you can use it on file reader.

#8

Can you please show me an example of what you mean?

#9

Okay I have that working correctly, one last question how do you get a timestamp to be populated on the end of a file?

#10

File Writer Snap - File property* ‘sample’+Date.now().toLocaleDateTimeString({“format”:“yyyy-MM-dd HH:mm”})+’.json’

For Reference.

https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1439344/Date+Functions+and+Properties