Forum Discussion

sathishmadavali's avatar
sathishmadavali
New Contributor II
3 years ago

How to load Today Date Files from s3 Bucket to SLDB

Hi, I am getting the files from amazon s3 bucket every day, I want to load today date files from s3 bucket to SLDB.Any one can help me on that. Thanking you in advance. Thanks M SATHISH
s3
Aleksandar_A's avatar
Aleksandar_A
Contributor III
3 years ago

Hi @sathishmadavali,

One way to filter only today’s files is to browse the S3 Bucket using the S3 Browser and after that to use a Filter Snap with the following expression:

Date.parse($lastModified) >= Date.parse(Date.now().toLocaleDateString())

If your file pool in S3 is growing at increasing rate you might consider another approach.

Let me know if this helps you.

BR,
Aleksandar.

sathishmadavali's avatar
sathishmadavali
New Contributor II
3 years ago

Hi,
@AleksandarAngelevski , the output of the filter snap is in csv format, by using which snap we can read the csv data .after getting the today file.

thanking you
m sathish

  • Aleksandar_A's avatar
    Aleksandar_A
    Contributor III
    3 years ago

    Hi @sathishmadavali,

    If you are using the S3 Browser Snap then the output is as in the picture below:

    You will have to use the File Reader Snap with the appropriate S3 Account and you will need to pass the Path value from the Filter Upstream.

    File Reader Snap:

    Let me know if this helps.

    BR,
    Aleksandar.

      • sathishmadavali's avatar
        sathishmadavali
        New Contributor II
        3 years ago

        Hi,

        how to browse before 2 days file?, its mean today is 14 june ,i want to browse12 june files