Snaplogic Triggered Task using an OnPremises URL through Load Balanced Groundplex Nodes

[dshen]

This article describes how a triggered task is invoked in the SnapLogic Elastic Integration
Platform using an OnPremise URL through load balanced Groundplex nodes.

Assume that your organization has a SnapLogic Groundplex provisioned with 3 nodes. When an OnPremise URL is exposed for a triggered task, it will automatically suggest the hostname of one of the nodes that belongs to the Groundplex.

e.g., https://GP-Node1:<port>/api/1/rest/feed/<RELATIVE_PATH_TO_TASK>/

To provide redundancy across all nodes in the Groundplex when a triggered task is invoked, a load balancer can be placed in front of the Groundplex nodes. When a load balancer is setup and configured, Snaplogic will use the load balancer in the auto-generated OnPremise URL.

e.g., https://GP-LB:<PORT>/api/1/rest/feed/<RELATIVE_PATH_TO_TASK>/

The following diagram describes the flow of network requests made when remotely executing a triggered task using a load balancer OnPremise URL.

1. A remote client invokes the triggered task using the OnPremise URL that points to the load balancer (e.g., GP-LB).
2. The load balancer forwards the request to an active groundplex node. GP-Node1 is selected for the purpose of this example.
3. The groundplex node that receives the triggered task request asks the Control Plane on which node the task should be executed.
4. The Control Plane forwards the request to an active groundplex node. GP-Node2 is selected for the purpose of this example.
5. The triggered task now prepares to be executed on GP-Node2. An HTTPS connection is created between GPNode-1 and GP-Node2 to enable data to be streamed between the nodes.
6. The data is read/write from/to the end points.
7. The response message is sent though GP-Node1 then GP-LB (load balancer) back to the caller.

[sanjaynayak]

@cjhoward18 Really cache part is causing issues.
Use case 1:
Since we have couple of nodes and task got executed by node1 and we are changing account info and 2nd request is telling invalid account info.
Use case 2:
Rest outh2 account token is expired and ideally system should regenerate it after expired, can we check where expire time we are caching.

It would be grate help.

[sanjaynayak]

@cjhoward18 ,

One of the special use case for caching issues , after restart it worked but in prod we don’t prefer to restart the server. Any suggestion will be helpfull.

Use Case:

I am experiencing a strange behavior on Prod. I had an error in one of the pipelines. I have fixed the pipeline and deployed the task. The error still persists. I have deleted the task. However, it is still being called with old version when I call the task from Postman.

[bojanvelevski]

@dshen @cjhoward18 @tlikarish @PSAmmirata 

Hello gentlemen.

This is a very interesting subject! Reading this I can understand that we practically have two balancing processes here, one from the Load Balancer itself, and the other one from the Control Plane. What I couldn't understand was, what is the criteria when the Control Plane is deciding on which node it should execute the request. The Cache is mentioned throughout the comments, does that mean that the Control Plane is making the decision based on which node the pipeline is already cached? And if yes, does that mean that, if the node which received the request, has the pipeline cached on itself, will execute the request and it won't pass it on to some of the other nodes?

Thank you,

Bojan