S3 File as Email Attachment

[jskrable]

Hello,

As part of an error handling pipeline, I am attempting to write a detailed error file to an S3 bucket, and then attach that file to the email notification at the end of the error pipeline to alert users of the issue.

I was able to do this successfully when writing the file to snaplogic’s native file system, but the email sender snap is failing consistently when attempting to grab the S3 file. The error message is:

Cannot find AWS access key ID in URL
Resolution:
Please enter valid URI path.
Reason:
URI syntax error.

For the attachments field, I have something like this:

s3:///buistintegration/test_errors_12082017.json

I tried adding the AWS Access Key and Secret in a few different combinations before the bucket name, with no luck.

Has anyone attempted this before? Is it possible?

Any input is appreciated.

Thanks,
Jack Skrable

[chenry]

Hi Jack,

So you were able to write the file, but the issue is when you try to retrieve the file in the next Snap and attach it to an email?

Are you using a FileReader Snap to read the file back out from S3? That would take your AWS keys and credentials, and handle the URL crafting for you.

Thanks,
-Charlie

[chenry]

Hi Jack,

I re-read your post, and saw that you are using the Email Sender Snap.

I created a pipeline to test a similar scenario, and I have a couple solutions for you!!!

To isolate the flow, in my pipeline, I use a FileReader Snap to read a file from S3 (this assumes that I have previously written one to S3, as in your scenario). This Snap takes my AWS credentials, and my file handle (which is of the format: s3:///bucketname@regionname.amazonaws.com/path/FileName)

I successfully read this file, but this of course produces a binary output. Next, I connect a BinaryToDocument Snap onto the FileReader. The encoding I choose is “NONE” This converts the File data into a platform specific characterset and loads it into a Java String.

I next connect a Mapper Snap to the BinaryToDocument. After I validate the (soon to be completed) pipeline, I then can see the generated source schema, which shows me “content” from my BinaryToDocument Snap. I mapped this to an output variable called “$body” although I probably could have just later referenced “$content”, but that was giving me troubles.

Next I connect my Email Sender Snap to the Mapper. I configure the account, etc etc etc, and in the body, I click the “=” sign and as my expression, I use “$body”. Now once I validate and execute the pipeline, my String data within the S3 File is emailed to me successfully!!!

Now, this was my proof of concept. Instead of using the FileReader…for your scenario I think the following would be more efficient:

I am assuming you are using an S3FileWriter to write your test_errrors*.json file to S3. In lieu of using the FileReader after writing the file, you can add an output view to your S3FileWriter, as per:
https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1439090/S3+File+Writer

That would produce a document output, which could then be used in the Mapper that I described, to map the content to “$body” and consume that in the Email Reader Snap.

Now, you may already be consuming the outputView for your S3FileWriter Snap, but fear not; there is a way to handle this as well: You can connect the output of the S3FileWriter to a Router Snap (found in Flow Snappack). This has 2 outputViews by default. You can place this in between the S3FileWriter, and the Mapper. Set the expressions for the Router Snap to true. This will make it copy the output from the S3FileWriter to the new pipeline junction, which maps the content, and emails it…along with any former junction.

I hope this was helpful. Please reach out if you need any more info on this!
Thanks,
-Charlie

[jskrable]

Hi Charlie,

Thanks for your detailed response. I’ve been able to manipulate the body of the email with a lot of flexibility, but my question revolves around using the attachment feature to attach an S3 file onto the email.

My pipeline is already writing the pertinent error data into the body of the email, but I want to be able to keep a more detailed record of the error and stacktrace in a file attached to the alert message. This is where I ran into trouble attaching an S3 file, while I’ve been successful using a file from the native snaplogic file system.

Is this something that is possible? Or does the email sender snap only support files from snaplogic?

Hope this clears up my question.

Thanks,
Jack

[chenry]

Hi Jack,

You’re very welcome for the information! At this time, our documentation does not specify that we support AWS S3 protocol within the Attachment field in the Email Sender Snap.
https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1438208/Email+Sender

I am not sure if such an enhancement is in the roadmap, but I could connect you with the right folks who can provide more information on this.

In the meantime, the example I described (parsing the file contents and inserting that to the email body) could be used. Please let me know if you would like to get in touch with our PM team.

Thank you for your time and patience.
-Charlie