Design Approach for Downloading SLDB saved files using smb

[SusanB]

Hello, looking for input regarding best approach to automate the manual download of various types of files (.csv, .xlsx, .zip) which are saved in the same project folder.
We want to keep them available in Snaplogic but also download to another server on certain days of the month.
File Operation /Copy does not seem like an option.
Thanks!

[SpiroTaleski]

@SusanB

Having a scheduled process/task that will run on certain days of the month, scanning the SLDB project/directory for a files(using Directory Browser Snap), File Reader Snap(to read the files), and File Writer Snap(to write the file to target system), will work, but also you should think of a logic where once copied files from SLDB to not be picked again in the next execution.

Regards,
Spiro Taleski

[SusanB]

Hi Spiro,
Thanks for your additional input. Luckily the filenames already include a timestamp.
Knowing this, the new pipeline will include a parameter of the list of month days that need to have files copied. IE “1,15” (for the first and fifteenth of each month).
The pipe will have a router that compares today’s month date to the parameter list. If today is not in the parameter’s list of values we exit. If it is we perform the copy steps:
Find and filter the list of files that contain today’s date, route them for zip vs non zip files, read and write (adding the option to Overwrite and/or Ignore if the file is already in the target directory. Currently zip file is having issue with ignore - so we’ll just overwrite those).
We will then schedule the task to run daily.
When needed for special requests we can easily add another temporary scheduled task for other days.
As time allows, I may attempt to make this more flexible by adding a parameter for specific file name prefixes.
For now this should eliminate the need for a human to manually copy files.