mohamadelmardin
New Contributor III
Joined 9 years ago
User Profile
User Widgets
Contributions
Re: Best practices to publish to a AWS SNS topic
@aleung Just realized I never shared back my solution how I solved to publish to SNS topic directly from SnapLogic. I created this utility pipeline that leverage Python script to invoke AWS Java SDK to do the job. I am setting the AWS secrets and other configuration such as topic name as pipeline parameters but they could be loaded from anywhere. Attached is a PoC pipeline. Publish to SNS Topic_2019_01_04.slp (7.1 KB) I believe the AWS SDK are already installed on the groundplex node by default but if not you can download them from AWS site and drop them in the JVM library where the snaplex is running.2.2KViews0likes2CommentsRe: Parsing XML Array to JSON array is inconsistent
mohamadelmardini: [].concat[ element ] First you have to use the out-of-box XML parser snap and configuring it should be easy as described in SnapLogic documentation. Then the hard part begins after the incoming XML input parsed into JSON is to navigate through the nested structure and map any array or non-existing elements. The best way to do that is to use a series of Mapper snaps and in each mapper you have to use expression language statements which is a subset of JavaScript. To read more about expression language please check the following link: https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1438042/Expression+Language+Overview For example to cast an parsed XML element in JSON into an array type you would use the following expression: [].concat[ element ] Please refer also to the beginning of this thread for other examples.4KViews0likes0CommentsRe: Parsing XML Array to JSON array is inconsistent
@nshruthi No I didn’t. Unfortunately SnapLogic doesn’t have an out-of-box solution in the XML parser to handle XML array transformation gracefully. I have an enhancement request opened with support but I haven’t heard any resolution on it yet and still waiting…7.3KViews0likes3CommentsConfiguring Amazon MQ credentials in RabbitMQ account
I am trying to connect to Amazon MQ broker from SnapLogic to consume messages by using the RabbitMQ account since it is the only option in SnapLogic that supports amqp protocol. When I enter the Amazon MQ credentials and click on validate I get the error below: Failed to validate account: An error occurred while creating the connection Cause: connect timed out (Reason: Invalid credentials.; Resolution: Please verify if the provided credentials are valid ) I think the connection to Amazon MQ should work fine even if I am using the Rabbit MQ snap account because both of them are compliant with amqp protocol. Can anyone advise me if I am doing anything wrong or if it is even possible to establish this connection? Do I have to specify the optional security fields such as UseTSSv1.2, keystore, etc…?3KViews0likes2CommentsRe: How to use 3rd party python libraries in python Script
Following up on my prevoius question, I guess I found the answer on this link: http://docs.aws.amazon.com/sns/latest/dg/using-awssdkjava.html But feel free to add to it or provide a Python code snippet for SNS if I missed anything…2KViews0likes0CommentsRe: How to use 3rd party python libraries in python Script
This is an excellent post cause we are running into similar problem with integrating to different AWS services such as publishing to SNS topic and we are trying to install Boto3 on the JCC node so hopefully the python script will pick it up. Apparently from this post installing python Boto3 won’t work and we need to and download AWS JAVA SDK instead and invoke it from Python script. This is a good solution to us but we need to know the java import libraries for SNS. Can someone help us by posting a similar code snippet but for publishing to SNS instead of S3 using the JAVA sdk from Python? What are the import libraries we need?2KViews0likes0CommentsRe: Best practices to publish to a AWS SNS topic
@aleung So you are referring me to use the CLI AWS SDK to integrate with SNS at runtime I have two questions there: I was able to download and run commands with AWS CLI on my local desktop machine but not sure how I would execute this command from my pipeline and after I set up and download CLI on groundplex. You mentioned Utilities folder in Box but were can I find it? Are you referring to Box (online file sharing service) snap pack or something else that I am completely out of context on? Also where is that Utilities folder located? Can I use another AWS SDK instead of CLI such as JavaScript or Python? The reason I mentioned these two other SDKs because I could leverage a script snap that supports any of these two languages and call the SNS API there. I am fine with the CLI if it works but I am just hinting for other suggestions if possible.3.6KViews0likes4CommentsRe: Best practices to publish to a AWS SNS topic
Thank you @aleung but this doesn’t address my issue and question. I don’t have a problem publishing and consuming messages from SQS. My problem is only with publishing to SNS topic since there isn’t snap to handle SNS topic publishing. I was wondering if there is a workaround either by using REST or javascript snaps to achieve that purpose.3.6KViews0likes6CommentsBest practices to publish to a AWS SNS topic
SnapLogic currently doesn’t have a dedicated snap pack for AWS SQS and SNS integration. Though we can use JMS consumer/producer snaps to configure it for SQS seamlessly however there is no mechanism to do that for SNS. I guess the only possible way to publish to an SNS topic within a pipeline is use the REST api via GET/POST snaps. Is that true? What are the configuration steps required for security to send access and secret keys with the REST snap in order to do that? where should be keep these security tokens in the http parameter or http header? how can we further protect them? Please provide best practices with working demo if any.11KViews0likes12CommentsHow to configuring JMS snap or REST snap to integrate with SQS FIFO queue
I am trying to specify for my endpoint an SQS queue FIFO type. I don’t care if I am using the REST snap or JMS snap but I am reaching a dead end when using an SQS account for FIFO queue in JMS snap because SQS account supports only STANDARD SQS queue not FIFO according to the link below: https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1439291/JMS+Account Is there a workaround to specify a FIFO SQS queue or do I need to request a feature enhancement from SnapLogic support?1.3KViews0likes0Comments