โ04-15-2022 03:37 AM
Hi Team,
I have designed pipeline to make recursive calls using pipeline execute snapโฆin one scenario i got an error at the 64th iteration tat it โโcannot execute any more child pipelinesโโ and i found in pipeline execute snap documentation that it cannot exceed more than 64 depth b4 they begin failingโฆ
But wen i executed the same pipeline after a weekโฆits taking 150 iterationsโฆhow is this possibleโฆ
โ04-15-2022 04:48 AM
@Noushin - to my knowledge, 64 is still the limit. I just created the attached pipeline to test the depth and 64 levels completes successfully but 65 fails. This was tested in 2 different orgs, so Iโm fairly confident that this threshold has not changed.
Test Recursion_2022_04_15.slp (9.9 KB)
โ04-15-2022 09:56 AM
Hi @koryknick - Please find the below attached screenshots where both error at 64th iteration and also the iterations that have reached 99 is attached
second image has gone until 99th iterationโฆ
โ04-15-2022 10:29 AM
@Noushin - letโs take a step back and make sure we are both using the correct terminology.
Recursion would be a pipeline that calls itself via Pipeline Execute - maximum depth is 64 calls deep in the stack of pipelines. Recursion is typically used to perform a task based on the current input value until a condition is met. For example, you may need to use recursion to get the full directory tree of a file system, so you get the listing of the current directory and then recursively call the pipeline for each directory to get the file list of each child directory. Recursion in Dashboard looks like this:
Iteration is calling a child pipeline by passing input documents in the stream to perform operations on each document. This is the typical (and recommended) pattern of calling the Pipeline Execute. Iteration in Dashboard looks like this:
Based on the error I see in your original thread, I believe you mean that previous executions only allowed 64 iterations (not recursion) and would indicate that something was consuming too many resources on the Snaplex node to allow any more child pipelines to execute. You may want to work with your SnapLogic CSM to look at your capacity planning. There is no limit to the number of child pipeline executions but you would lose visibility of the statistics after a number of iterations.
โ04-15-2022 10:44 AM
Hi @koryknick - Even I am making recursive calls using pipeline execute snap with condition until all records from the soap services are extracted i call same pipeline in recursive modeโฆAnd in the image i have dedelected the group pipelines checkbox hence its not on tree structure, please find the below image which depicts the recursive call.