skidambi
Former Employee
Joined 9 years ago
User Profile
User Widgets
Contributions
Re: Architecture: Pipeline Execution Mechanism - Some Questions
Hello - Based on your question, with Snaplex A and Snaplex B in place; if the parent pipeline is getting executed in one of the nodes of Snaplex A, and I am assuming you are calling the child pipeline using the Pipeline Execute snap. The Pipeline Execute snap has the following three options for the developer to choose from (on the field labeled Execute On): SNAPLEX_WITH_PATH When you choose this option, another field Snaplex Path would appear. NOT PROVIDING any info on this Snaplex Path field, would result in the (chosen) child pipeline getting executed in the SAME NODE as the parent. For example, if parent pipeline is executing on a node in Snaplex A, the child pipeline specified in the Pipeline Execute snap would execute in the same node where the parent is being executed. Providing a valid Snaplex path, would result in the child running in any of the available nodes in the Snaplex provided. LOCAL_NODE Child pipeline gets executed in the same node (of the same Snaplex) where the parent is being executed. LOCAL_SNAPLEX Child pipeline gets executed in the same Snaplex where is parent is being executed. Note that this only means the same Snaplex but the child and parent may get executed in different nodes depending upon how busy each of the nodes in that Snaplex are.3.4KViews0likes0CommentsRe: How to identify the IP address of a Cloudplex Node?
Hello Supratim - Thank you for your response. This method of identifying the Cloud Node’s IP address is deprecated now. Please refer to the post by cstewart during March 2017 in this thread above. I am pasting it for your kind reference below (This has been the case since April 2017): > Bear in mind that a cloudplex is an entirely replaceable node. It may not stick with that IP address. The upcoming maintenance on the 8th April will have all cloudplex nodes conform the IPs in the range: 136.179.34.0/25, as advised, all destinations should be updated with that info prior to the maintenance on that day. From then on the IP range is a CIDR (136.179.34.0/25)!2.3KViews0likes2CommentsArchitecture: Pipeline Execution Mechanism - Some Questions
How does the SnapLogic Control Plane identify a specific node in a Snaplex for executing a pipeline? How does it identify if a node is busy? Is it only checking one time for the node’s availability? This is a continuous process: When a node is fired up (started), it establishes a connection with the SnapLogic Control Plane over a non-persistent outbound connection via port 443. This is in essence, a registration process based on the configuration file (Basically the node tells the Control Plane, “I am a node, that belongs to a particular plex which is an asset within a particular org”). Once this is done, the node and Control Plane start communicating with each other via web socket connections. The node keeps sending information such as heart beat, process details to the Control Plane continually. Based on this info, the Control Plane is kept aware of the number of active threads in each node continually (On the Dashboard, you can get to the Snaplex-> Node-> Additional Information). So this is not a one-time activity but an ongoing activity. Among the nodes within a Snaplex, the Control Plane makes a choice of that node which has the least number of active threads (In other words, that is the “least busy” node), to send the pipeline executions .5.9KViews3likes3CommentsRe: Anyway to recover deleted pipelines
Hello Daisy - I presume this is not your “production” org? Unfortunately there is not any way to recover the pipelines in SnapLogic since these are not persisted anywhere in SnapLogic (like a Trash / Recycle Bin). Would these pipelines (that you deleted) have a presence in your production org? If yes, the best way is to check with the admin of your production org to provide you with a copy.2.7KViews0likes0CommentsAggregation of IDs for IN clause in a Delete DB Snap
When a Database Delete (Example: SQL Server Delete) snap is used, the snap goes through each of the entries in the incoming document and makes one roundtrip to the database to delete the pertinent record. Whilst it could be argued that this is not the efficient method, the snap does provide a great relief in providing the user with the ability to be able to use the IN keyword. Hence, all of the pertinent record info necessary for the delete operation can be aggregated using a simple procedure (see attached pipeline). This way all of the deletes can be performed in one step using the delete snap. Steps Involved: Concatenate all of the pertinent IDs from the incoming document using an aggregate snap (Use the UNIQUE_CONCAT function and Unsorted option for Sorted Streams, Example: $KeyID to $KeyIDs in the sample pipeline provided). Use a mapper snap and provide the following expression to the target path (expression is based on the example provided in the sample pipeline): “(”+$[‘$KeyIDs’].replace(/|/g,“,”)+“)” The above expression converts the IDs to a comma separated list of values bound by parenthesis (as a string) as output. Feed the output of the mapper snap to the delete snap. Aggregate-IDs_2017_02_26.slp (5.5 KB)2.4KViews2likes0CommentsHow to identify the IP address of a Cloudplex Node?
Following are the steps to be followed whilst trying to identify the IP address of a Cloudplex Node: Login to https://elastic.snaplogic.com Go to Dashboard and change the Org to the customer’s org to which their Cloudplex is associated with. Click on the Cloudplex so it shows the associated nodes and note down their names. Copy and paste the following link on the same browser instance: https://elastic.snaplogic.com/api/1/rest/slserver/ccinfo/<> Example: https://elastic.snaplogic.com/api/1/rest/slserver/ccinfo/Target Please make sure you have installed a JSON View and JSON Formatter extensions installed for your browser (if Chrome or equivalent if Firefox), prior to executing this step. Search for the node name in the browser. Look for a entry called “uri” and the value it has. An example of how it would look like would be http://prodxl-jcc261.fullsail.snaplogic.com:8090 Remove the “http://” and “:8090” from the above string and copy the rest. In this example it would be prodxl-jcc261.fullsail.snaplogic.com Open a new terminal (or command prompt if Windows) and execute the following command: ping prodxl-jcc261.fullsail.snaplogic.com Note that the IP address is returned.9.5KViews2likes9Comments