How to load Today Date Files from s3 Bucket to SLDB

[sathishmadavali]

Hi,
I am getting the files from amazon s3 bucket every day, I want to load today date files from s3 bucket to SLDB.Any one can help me on that. Thanking you in advance.

Thanks
M SATHISH

[AleksandarAngel]

Hi @sathishmadavali,

One way to filter only today’s files is to browse the S3 Bucket using the S3 Browser and after that to use a Filter Snap with the following expression:

Date.parse($lastModified) >= Date.parse(Date.now().toLocaleDateString())

If your file pool in S3 is growing at increasing rate you might consider another approach.

Let me know if this helps you.

BR,
Aleksandar.

[sathishmadavali]

Hi,
@AleksandarAngelevski , the output of the filter snap is in csv format, by using which snap we can read the csv data .after getting the today file.

thanking you
m sathish

[AleksandarAngel]

Hi @sathishmadavali,

If you are using the S3 Browser Snap then the output is as in the picture below:

You will have to use the File Reader Snap with the appropriate S3 Account and you will need to pass the Path value from the Filter Upstream.

File Reader Snap:

Let me know if this helps.

BR,
Aleksandar.

[sathishmadavali]

@AleksandarAngelevski
Thanking you