Passing document from parent pipeline to child pipeline

[Nagaraju_Julaka]

Hi Team,

I have a requirement to pass the document from parent pipeline to child pipeline, in child pipeline it has only one snap(Mapper Snap).

Document which will be passing from parent pipeline should go as a input to mapper snap in child pipeline and provide the mapper output to parent pipeline back which will be using in parent pipeline for further processing of data.

Can someone help on the above requirement.

[ptaylor]

Using Pipeline Execute will definitely be slower than putting the Mapper directly in your pipeline, due to the overhead of Pipeline Execute. Hopefully, the performance with Pipeline Execute will be adequate. But make sure you check “Reuse executions to process documents” on the Pipeline Execute to get the best performance. This will launch just one child pipeline (or N child pipelines if you set Pool Size = N) and feed all of the input documents to that instances (or those N instances). Otherwise, it will create a new pipeline instance for every single document, which is FAR slower.

[skatpally]

@ptaylor - Sorry I missed the Re-use Checkbox. We did see quite a performance improvement with a pipeline on setting greater pool size and re-use checkbox checked.

[ptaylor]

That’s good. Yes, I didn’t mean as a general rule. It very much depends on what the child pipeline is doing. If it’s just a Mapper, I still think it’s unlikely you’d see much improvement. In my test, increasing the Pool Size increased the time.