06-08-2023 11:35 PM
Hi,
I am getting the files from amazon s3 bucket every day, I want to load today date files from s3 bucket to SLDB.Any one can help me on that. Thanking you in advance.
Thanks
M SATHISH
06-09-2023 12:26 AM
Hi @sathishmadavali,
One way to filter only today’s files is to browse the S3 Bucket using the S3 Browser and after that to use a Filter Snap with the following expression:
Date.parse($lastModified) >= Date.parse(Date.now().toLocaleDateString())
If your file pool in S3 is growing at increasing rate you might consider another approach.
Let me know if this helps you.
BR,
Aleksandar.
06-14-2023 05:07 AM
Hi,
@AleksandarAngelevski , the output of the filter snap is in csv format, by using which snap we can read the csv data .after getting the today file.
thanking you
m sathish
06-14-2023 05:13 AM
Hi @sathishmadavali,
If you are using the S3 Browser Snap then the output is as in the picture below:
You will have to use the File Reader Snap with the appropriate S3 Account and you will need to pass the Path value from the Filter Upstream.
File Reader Snap:
Let me know if this helps.
BR,
Aleksandar.
06-14-2023 06:06 AM
@AleksandarAngelevski
Thanking you