Parent-child pipeline execute

[pmancevski]

Hello all,

I’m trying to understand one functionality for “pipeline execute”. Let say we need to have multiple executions in the child process and the pool size is 1. My question is does pipeline execute is scanning the project environment before initializing the instances(child processes) or the project environment is scanned only once when parent pipeline is initialized?

Thanks,
Pero M.

[tlikarish]

Hi Pero,

Could you be more specific about what you mean by “scanning.” From the context, I’m interpreting this question to mean: “When is the child pipeline loaded/initialized?” Is that what you’d like more information about?

Best,
Tim

[pmancevski]

Hi @tlikarish,

Sorry for my bad explanation.
Yes, when is the child pipeline loaded. When parent is initialized or on every input document on pipeline execute.

Thanks,
Pero M.

[tlikarish · Accepted solution]

Pero,

No worries – just want to make sure I answer the right question before diving in.

To give an idea of the flow, there are two cases to consider. The pipeline property can be an expression or a static property. In the case that the property is static, the child pipelines are loaded during the prepare and bundled together with the parent. If they are expressions, then the expression is not evaluated before execution begins. After the expression is evaluated, then the Snaplex node will fetch the child pipeline for execution. The child pipeline is fetched once [1] during the execution and that version of the pipeline is used for the remainder of the parent’s execution.

This generally holds true for other asset types as well like accounts. The platform tries to statically link what it can prior to pipeline execution, but has to handle the more dynamic elements separately.

Hope this helps answer your question and let me know if I can clarify anything else or help with a more specific issue.

[1] There was a bug prior to 4.32 where the child pipeline might be loaded more than once, so you may notice different behavior on the previous version.

[pmancevski]

Hi,

Thanks a lot, this is sufficient answer…