cancel
Showing results for 
Search instead for 
Did you mean: 

How to load Today Date Files from s3 Bucket to SLDB

sathishmadavali
New Contributor II

Hi,
I am getting the files from amazon s3 bucket every day, I want to load today date files from s3 bucket to SLDB.Any one can help me on that. Thanking you in advance.

Thanks
M SATHISH
s3 load

17 REPLIES 17

Aleksandar_A
Contributor III

Hi @sathishmadavali,

One way to filter only today’s files is to browse the S3 Bucket using the S3 Browser and after that to use a Filter Snap with the following expression:

Date.parse($lastModified) >= Date.parse(Date.now().toLocaleDateString())

image

If your file pool in S3 is growing at increasing rate you might consider another approach.

Let me know if this helps you.

BR,
Aleksandar.

Hi,
@AleksandarAngelevski , the output of the filter snap is in csv format, by using which snap we can read the csv data .after getting the today file.

thanking you
m sathish

Hi @sathishmadavali,

If you are using the S3 Browser Snap then the output is as in the picture below:
image

You will have to use the File Reader Snap with the appropriate S3 Account and you will need to pass the Path value from the Filter Upstream.

image

File Reader Snap:
image

Let me know if this helps.

BR,
Aleksandar.

@AleksandarAngelevski
Thanking you