cancel
Showing results for 
Search instead for 
Did you mean: 

Pipeline execute maximum recursion calls depth

Noushin
New Contributor

Hi Team,

I have designed pipeline to make recursive calls using pipeline execute snap…in one scenario i got an error at the 64th iteration tat it ‘‘cannot execute any more child pipelines’’ and i found in pipeline execute snap documentation that it cannot exceed more than 64 depth b4 they begin failing…

But wen i executed the same pipeline after a week…its taking 150 iterations…how is this possible…

6 REPLIES 6

koryknick
Employee
Employee

@Noushin - to my knowledge, 64 is still the limit. I just created the attached pipeline to test the depth and 64 levels completes successfully but 65 fails. This was tested in 2 different orgs, so I’m fairly confident that this threshold has not changed.

Test Recursion_2022_04_15.slp (9.9 KB)

Hi @koryknick - Please find the below attached screenshots where both error at 64th iteration and also the iterations that have reached 99 is attached

64_iteration failed

second image has gone until 99th iteration…

99_iTERATIONS

koryknick
Employee
Employee

@Noushin - let’s take a step back and make sure we are both using the correct terminology.

Recursion would be a pipeline that calls itself via Pipeline Execute - maximum depth is 64 calls deep in the stack of pipelines. Recursion is typically used to perform a task based on the current input value until a condition is met. For example, you may need to use recursion to get the full directory tree of a file system, so you get the listing of the current directory and then recursively call the pipeline for each directory to get the file list of each child directory. Recursion in Dashboard looks like this:
image

Iteration is calling a child pipeline by passing input documents in the stream to perform operations on each document. This is the typical (and recommended) pattern of calling the Pipeline Execute. Iteration in Dashboard looks like this:
image

Based on the error I see in your original thread, I believe you mean that previous executions only allowed 64 iterations (not recursion) and would indicate that something was consuming too many resources on the Snaplex node to allow any more child pipelines to execute. You may want to work with your SnapLogic CSM to look at your capacity planning. There is no limit to the number of child pipeline executions but you would lose visibility of the statistics after a number of iterations.

Hi @koryknick - Even I am making recursive calls using pipeline execute snap with condition until all records from the soap services are extracted i call same pipeline in recursive mode…And in the image i have dedelected the group pipelines checkbox hence its not on tree structure, please find the below image which depicts the recursive call.

recursion