Forum Discussion
Using Pipeline Execute will definitely be slower than putting the Mapper directly in your pipeline, due to the overhead of Pipeline Execute. Hopefully, the performance with Pipeline Execute will be adequate. But make sure you check “Reuse executions to process documents” on the Pipeline Execute to get the best performance. This will launch just one child pipeline (or N child pipelines if you set Pool Size = N) and feed all of the input documents to that instances (or those N instances). Otherwise, it will create a new pipeline instance for every single document, which is FAR slower.
@ptaylor - Sorry I missed the Re-use Checkbox. We did see quite a performance improvement with a pipeline on setting greater pool size and re-use checkbox checked.
- ptaylor5 years agoEmployee
That’s good. Yes, I didn’t mean as a general rule. It very much depends on what the child pipeline is doing. If it’s just a Mapper, I still think it’s unlikely you’d see much improvement. In my test, increasing the Pool Size increased the time.