Snaplogic Triggered Task using an OnPremises URL through Load Balanced Groundplex Nodes

[dshen]

This article describes how a triggered task is invoked in the SnapLogic Elastic Integration
Platform using an OnPremise URL through load balanced Groundplex nodes.

Assume that your organization has a SnapLogic Groundplex provisioned with 3 nodes. When an OnPremise URL is exposed for a triggered task, it will automatically suggest the hostname of one of the nodes that belongs to the Groundplex.

e.g., https://GP-Node1:<port>/api/1/rest/feed/<RELATIVE_PATH_TO_TASK>/

To provide redundancy across all nodes in the Groundplex when a triggered task is invoked, a load balancer can be placed in front of the Groundplex nodes. When a load balancer is setup and configured, Snaplogic will use the load balancer in the auto-generated OnPremise URL.

e.g., https://GP-LB:<PORT>/api/1/rest/feed/<RELATIVE_PATH_TO_TASK>/

The following diagram describes the flow of network requests made when remotely executing a triggered task using a load balancer OnPremise URL.

1. A remote client invokes the triggered task using the OnPremise URL that points to the load balancer (e.g., GP-LB).
2. The load balancer forwards the request to an active groundplex node. GP-Node1 is selected for the purpose of this example.
3. The groundplex node that receives the triggered task request asks the Control Plane on which node the task should be executed.
4. The Control Plane forwards the request to an active groundplex node. GP-Node2 is selected for the purpose of this example.
5. The triggered task now prepares to be executed on GP-Node2. An HTTPS connection is created between GPNode-1 and GP-Node2 to enable data to be streamed between the nodes.
6. The data is read/write from/to the end points.
7. The response message is sent though GP-Node1 then GP-LB (load balancer) back to the caller.

[ram3]

@tlikarish @cjhoward18 Thank you for this blog post. My understanding is, if we use the Cloud URL then there is no need to setup the “GP_LB” right ? I am saying this because, Control plane has the information of which node is available and least loaded (has resources) to run the task & also has information of whether the task is already cached in the node.

Additionally, do you discourage the triggering of tasks using Cloud URL for security reasons ? If so, how to disable the CloudURL. Thanks in advance

[PSAmmirata]

Thank you for the explanation. I have a follow up question. Does the entire request message, including HTTP headers, parameters, and body get sent to the Control Plane and forwarded to the node selected for execution? Or is data like parameters and body not sent to the Control Plane, but is sent directly from the initial landing node (GP-Node1) to the selected node (GP-Node2) over the HTTPS connection established in step 5? Putting it another way is the HTTPS connetion between GPNode-1 and GP-Node2 only used to send the response back from GP-Node2 to GP-Node1 or is it used for anything else?

[tlikarish]

Most of the headers and parameters will get sent to the control plane, but a request body is not. Basically anything that can be converted into a pipeline parameter could be passed up as part of the request, but any data that is passed into an input view (binary or document) will be transferred between CC nodes themselves. There are some additional optimizations that have been made to try and avoid the control plane portion of this flow as well, so sometimes it’s even possible to avoid those steps as well and everything is handled locally between plex nodes.

So in worst case, the headers and parameters may be passed up to control plane, but the body is passed between nodes.

[sanjaynayak]

Regarding point #1, any one has any info.

1. A remote client invokes the triggered task using the OnPremise URL that points to the load balancer (e.g., GP-LB).

What authentication mechanism we can use between remote client and Onpremise ground URL since ground url has no token like cloud url we have bearer token.

[tlikarish]

Both cloud and ground triggered tasks support the bearer token and basic auth for authentication.

The new API manager feature allows one to use additional authentication policies.