Recursive Pipeline Solution

#1

I am curious about using recursive pipelines. I know the documentation says that we should not use pipelines in a recursive manner but for some solutions, I think it makes sense. I have created two pipelines: Recursion Parent and Recursion Child.The Recursion Parent passes in a simple js variable, $test which is set to 2 in this case. The Recursion Parent then calls the Recursion Child pipeline which then increments $test by 1 while subsequently decrementing the pipeline parameter param by 1. Once param == 0 then the Child pipeline stops and returns the value of $test and a status, in this case “completed”.

Recursive Parent Input

Recursive%20Parent%20Input
Recursive Parent Input.PNG893×651 34.2 KB

Recursive Parent Output

Recursive%20Parent%20output
Recursive Parent output.PNG764×591 20.3 KB

Recursive Child Input

Recursive%20Child%20input
Recursive Child input.PNG1230×705 38.1 KB

Recursive Child Pipeline Execute

recursive%20child%20execute
recursive child execute.PNG638×697 32.1 KB

My question/point to this is: Why is recursion not supported? It obviously works with a minor work around.

#2

Recursion is not supported because it’s too easy to lose control of the execution and have it start spawning child executions and overloading the Snaplex.

Can I ask what you’re trying to achieve?

#3

My team wants to be able to get a list of pipelines that call other pipelines. For example:

Pipeline A calls Pipeline B. Pipeline B calls Pipeline C.

Using recursion is the most elegant, and in my opinion, the most correct way to traverse this type of list. Furthermore, we know that there is a limit to how many children can be called, for us it is about 50. For traversing pipelines, we will never have more than 4 or 5 nested children at a time.

#4

Isn’t recursion where something calls itself?

You should be able to have A that calls B, and then B that calls C.

Sounds like maybe you’re just trying to implement looping (until param == 0)?

#5

Not sure if it’s “legit” from SnapLogic’s perspective, but you can implement looping between parent/child pipelines. Have the child pipeline exposed via triggered task, then have parent pipeline use REST GET snap to call the child pipeline triggered task, and use the REST GET’s pagination features (Has Next & Next URL) to evaluate some criteria and re-GET the child pipeline until criteria is false.