Snaplogic Triggered Task using an OnPremises URL through Load Balanced Groundplex Nodes

[dshen]

This article describes how a triggered task is invoked in the SnapLogic Elastic Integration
Platform using an OnPremise URL through load balanced Groundplex nodes.

Assume that your organization has a SnapLogic Groundplex provisioned with 3 nodes. When an OnPremise URL is exposed for a triggered task, it will automatically suggest the hostname of one of the nodes that belongs to the Groundplex.

e.g., https://GP-Node1:<port>/api/1/rest/feed/<RELATIVE_PATH_TO_TASK>/

To provide redundancy across all nodes in the Groundplex when a triggered task is invoked, a load balancer can be placed in front of the Groundplex nodes. When a load balancer is setup and configured, Snaplogic will use the load balancer in the auto-generated OnPremise URL.

e.g., https://GP-LB:<PORT>/api/1/rest/feed/<RELATIVE_PATH_TO_TASK>/

The following diagram describes the flow of network requests made when remotely executing a triggered task using a load balancer OnPremise URL.

1. A remote client invokes the triggered task using the OnPremise URL that points to the load balancer (e.g., GP-LB).
2. The load balancer forwards the request to an active groundplex node. GP-Node1 is selected for the purpose of this example.
3. The groundplex node that receives the triggered task request asks the Control Plane on which node the task should be executed.
4. The Control Plane forwards the request to an active groundplex node. GP-Node2 is selected for the purpose of this example.
5. The triggered task now prepares to be executed on GP-Node2. An HTTPS connection is created between GPNode-1 and GP-Node2 to enable data to be streamed between the nodes.
6. The data is read/write from/to the end points.
7. The response message is sent though GP-Node1 then GP-LB (load balancer) back to the caller.

[sanjaynayak]

Thanks @tlikarish, it is helpful and will cover so many use cases.

[sanjaynayak]

@tlikarish , New Api Manager policy is applicable for cloud url or ground url or both for trigger task.

I have applied Request Size Limit policy for one of the trigger task, seems to be not working for cloud url.

[cjhoward18]

Hi @sanjaynayak

API Policies can be enabled on tasks that are set to run on either a cloud plex or a ground plex, however, all APIM functionality, including the policies, is only available in the ground URL invocation of a triggered task. Any request sent to the control plane(cloud URL) instead of the plex/jcc will not have policies applied, or be part of the APIM feature set(dashboard, policies, etc).

[sanjaynayak]

@dshen ,any one suggest is it real workflow, it seems like we found some gap.

1- GP-LB is receiving the request from client, then as part of load balance we are telling to go to let sey-GP-Node1 in this example.
2- From GP-Node1 to control plane flow(3) is to get the pipeline(meta data) to be execute or what ?
3- If yes then control plane should chose GP-node1 to execute the pipeline since it was asked by GP-LB.

Can any body explain the actual design.

[cjhoward18]

Hi,

Step 2 only happens if the task is not already cached on the node itself.

Step 3, the control plane plays no role in choosing the node for execution in the Ground invoked case.
The communication is outbound from the GP Node to the control plane to fetch the depending assets for execution if they are not cached as stated before.