chenry
Former Employee
Joined 9 years ago
User Profile
User Widgets
Contributions
Re: Building the old custom snap
Thanks @dmiller ! Yes, the -U should force the update of your dependencies. It could be that either the Snappack Installer dependency that you have in your local repository is not compatible, or platform dependencies, or (very likely due to what I now see in the Stack trace) the Platform Jschema dependency may be incompatible. If you build with -U and force all of the dependencies to be analyzed, and re-downloaded, that should hopefully resolve the issue. Thanks all, -Charlie2.2KViews0likes0CommentsRe: Dynamic Drop Down Menu
Hi there, Looks like you’re attempting to put all possible subcategory selections into one drop down menu, and then hide certain ones. That could get tricky. You may want to do something more UI-tricky, and create two sub category drop down lists. One with animals, and one with cars. The default subcategory can be shown, if you default the main category to say…animals, and then if you make the main category selection, you can hide/unhide the other drop downs accordingly. That may do the trick, but then you have to be careful in your handling code, to key on the enums that you pull out of the subcategory based on the main category. I think the trickiest part here would be the hiding and unhiding of fields. Make sure that if you go this route, that both dropdown menus are the same size and placement, otherwise the UI might look meh, if you have components on the Snap popping in and out of view, and skewing the placement of other UI components. Thanks, -Charlie1.6KViews0likes0CommentsRe: Connect SFTP server to fetch multiple files
Hi Amar, Sorry for the delay. After reading about the two things that you want to (parse data from the filename and parsing the data) I had a conversation with a colleague to verify some of my thoughts, and get more advice. We may need to create two pipelines “A” and “B” in pipeline A We switch from MultiFileReader to DirectoryBrowser. This is because Directory Browser will give non binary data. The resulting document will contain fields like name, type, size, path. You could hook this into a PipelineExecute Snap. Which will execute Pipe “B” You can pass this snap the “path” field. In Pipeline “B” You can use a FileReader Snap to read the “path” field. Your file that gets returned, you said was a .txt file, although it is delimited with tabs and so should be able to be treated like a CSV file. So you should be able to hook up a CSV Formatter Snap to this File Reader. Then you should be able to process the data in this Pipeline “B”. You could also ramp up the pool size, and let the Pipeline Execute Snap run multiple executions in parallel. But for starters, let’s see how it runs with one thread. Thanks, -Charlie2.2KViews0likes0CommentsRe: Connect SFTP server to fetch multiple files
Hi Amar, glad you’ve had success so far! I experienced the same when I connect the BinaryToDocument to a Multi File Reader. However, if I connect it to a FileReader and specify one specific file, I get preview data. This may be a validation mode limitation. So the output of the MultiFile Reader is showing me JSON data with attributes about the files, including the file handle. The handle is in the “content-location” field. If you’ve hooked up your BinaryToDocument Snap… Depending on what you want to do with the documents, you’ll want to encode/decode differently. If you want to pass around raw bytes, then you can choose “BYTE_ARRAY” but if you want to do some parsing of your txt files, then you probably will want to use “NONE” encoding. What this is actually doing behind the scenes is pushing the data into a String format with the default charset of your node (likely UTF-8 depending on the file, but I digress). The next move is to hook up a Mapper Snap. In the expression field, enter a dollar sign (make sure the equals sign is checked). Same with the target path field. Now validate the pipeline one time. Might take a moment to run. When you open up the Mapper Snap (not the preview data) You can see the schema in the left hand side. Using this, you could map the “content-location” to some other “name” field that you should be able to grab from the output of the Mapper during the actual execution. This field could be keyed on as the input to the File Delete I believe. Let me know how this works out, and we can go further. Thanks, -Charlie2.2KViews1like1CommentRe: Connect SFTP server to fetch multiple files
Hi Amar, Let’s first start with the Multi File Reader Snap. If you create a new one, and then in the account tab, create a new account. For one of our internal test accounts, I chose basic auth. I am prompted for user name password, which I give. Next in the main settings tab of the Snap, I input the following under file/folder the format: sftp://hostName/DirectoryStructure/TargetFolder/ Then in the wildcard I put “.csv” And I am able to run successfully. If I look at my output as JSON, I can see “content-location” which includes the absolute path to the file name (including the file name). From this point, you should be able to use a BinaryToDocument Snap in order to operate on the document data. But let’s see how things go up to this point first. Thanks, and apologies for the delay. -Charlie2.2KViews1like1CommentRe: Connect SFTP server to fetch multiple files
Hi Amar, Sorry for the delay. I was thinking more about your scenario, and I think I have a better suggestion. Since you may not need to “poll” for the files, I am thinking a different Snap would be better. Say I want to boil the problem down to grabbing the files, and then deleting them…You could use a MultiFileReader Snap, with a file filter to get the desired types of file. The output will be a binary data stream. You could connect this to a BinaryToDocument Snap and then perform operations on that document. At some point you would need to use a Mapper to key on the filename and pass that to a FileDelete Snap. For the simpler case of just say…reading files and deleting them for proof of concept, you could use a DirectoryBrowser instead of the MultiFileReader. The DirectoryBrowser would return you attributes about all of the files that match the type filter (essentially a List in document format versus binary). Then you’d plug this into a Mapper Snap, to map the filename. Then you’d plug this into a Delete Snap where you configure it to read the file name from the mapped variable. I hope this helps you get further towards your goal! Thanks, -Charlie3.3KViews0likes3CommentsRe: Connect SFTP server to fetch multiple files
Hi Amar, I believe that the first challenge could be solved by using the FilePoller, and use the file filter option. This Snap will produce output containing the paths of the *.txt files, but not the actual files. After that, you could potentially use a ForEach Snap to process the path to the *.txt file. I need to confer with my teammates to see how one would go about deleting the files. Hope this gets you off to a good start! Thanks, -Charlie3.3KViews0likes1CommentRe: S3 File as Email Attachment
Hi Jack, You’re very welcome for the information! At this time, our documentation does not specify that we support AWS S3 protocol within the Attachment field in the Email Sender Snap. https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1438208/Email+Sender I am not sure if such an enhancement is in the roadmap, but I could connect you with the right folks who can provide more information on this. In the meantime, the example I described (parsing the file contents and inserting that to the email body) could be used. Please let me know if you would like to get in touch with our PM team. Thank you for your time and patience. -Charlie3.5KViews1like1CommentRe: How can you modify a reltio tenant mapping wide via a pipeline parameter?
Currently, this is targeted for the 4.12 release cycle, in order to ensure thorough and robust QA and UAT cycles. SnapLogic has a quarterly cadence on releases; however, prior to the release I believe that we have approximately a month of user acceptance testing in the UAT environment. As this is a new SnapPack, we want to focus keenly on quality cycles, and user feedback cycles; the more eyes on the product, the better. Please send me an email at chenry@snaplogic.com and I will introduce you to our Product Management team. They will be able to discuss the road map in detail; moreover, the process to get you up and running with a UAT environment. My apologies for the delay in this message. Looking forward to receiving your email. Have a great weekend! -Charlie2.5KViews0likes0CommentsRe: S3 File as Email Attachment
Hi Jack, I re-read your post, and saw that you are using the Email Sender Snap. I created a pipeline to test a similar scenario, and I have a couple solutions for you!!! To isolate the flow, in my pipeline, I use a FileReader Snap to read a file from S3 (this assumes that I have previously written one to S3, as in your scenario). This Snap takes my AWS credentials, and my file handle (which is of the format: s3:///bucketname@regionname.amazonaws.com/path/FileName) I successfully read this file, but this of course produces a binary output. Next, I connect a BinaryToDocument Snap onto the FileReader. The encoding I choose is “NONE” This converts the File data into a platform specific characterset and loads it into a Java String. I next connect a Mapper Snap to the BinaryToDocument. After I validate the (soon to be completed) pipeline, I then can see the generated source schema, which shows me “content” from my BinaryToDocument Snap. I mapped this to an output variable called “$body” although I probably could have just later referenced “$content”, but that was giving me troubles. Next I connect my Email Sender Snap to the Mapper. I configure the account, etc etc etc, and in the body, I click the “=” sign and as my expression, I use “$body”. Now once I validate and execute the pipeline, my String data within the S3 File is emailed to me successfully!!! Now, this was my proof of concept. Instead of using the FileReader…for your scenario I think the following would be more efficient: I am assuming you are using an S3FileWriter to write your test_errrors*.json file to S3. In lieu of using the FileReader after writing the file, you can add an output view to your S3FileWriter, as per: https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1439090/S3+File+Writer That would produce a document output, which could then be used in the Mapper that I described, to map the content to “$body” and consume that in the Email Reader Snap. Now, you may already be consuming the outputView for your S3FileWriter Snap, but fear not; there is a way to handle this as well: You can connect the output of the S3FileWriter to a Router Snap (found in Flow Snappack). This has 2 outputViews by default. You can place this in between the S3FileWriter, and the Mapper. Set the expressions for the Router Snap to true. This will make it copy the output from the S3FileWriter to the new pipeline junction, which maps the content, and emails it…along with any former junction. I hope this was helpful. Please reach out if you need any more info on this! Thanks, -Charlie3.5KViews0likes3Comments