Forum Discussion

graham-onpoint's avatar
graham-onpoint
New Contributor II
6 years ago

Writing files with Child Pipeline - spawning multiple children?

I have a simple debugging Pipeline that I would like to insert at various points in my main Pipeline:

Two parameters are sent to the debugging pipeline:

The router uses the DEBUG value to see if it needs to send output to the Excel formatter and File Writer.

Here’s an example of how this is being called from the parent/master Pipeline, it reads the DEBUG parameter from the Pipeline level:

When I add this into the parent/master pipeline it never finishes executing and it spawns a LOT of child pipelines according to the Dashboard. Files are being written to the system.

Here is the section of the parent pipeline where the debug calls are being made, there are only three inserted right now:

What is causing this to spawn so many processes and effectively freeze?

When inserted into the parent/main pipeline with copy/paste they work exactly as expected.

3 Replies

  • christwr's avatar
    christwr
    Contributor III
    6 years ago

    PipelineExecute will invoke the child pipeline for every document.

  • tstack's avatar
    tstack
    Former Employee
    6 years ago

    As @christwr mentioned, the default behavior in PipeExec is to spawn a child for every input document. It sounds like you want the opposite behavior, to start a single child that processes all documents received by the PipeExec. You can switch to that behavior by selecting the Reuse executions to process documents option.