Passing document from parent pipeline to child pipeline

[Nagaraju_Julaka]

Hi Team,

I have a requirement to pass the document from parent pipeline to child pipeline, in child pipeline it has only one snap(Mapper Snap).

Document which will be passing from parent pipeline should go as a input to mapper snap in child pipeline and provide the mapper output to parent pipeline back which will be using in parent pipeline for further processing of data.

Can someone help on the above requirement.

[ForbinCSD]

@skatpally, that’s brilliant!

[ptaylor]

One Scenario could be - If your mapper has Heavy Transformations then you can move it to child pipeline and have pool size set to greater than 1 to improve the performance.

Unlikely. In fact, it might make it slower, due to the overhead of launching child pipelines and distributing the load across them.

[Nagaraju_Julaka]

Thanks, It will help us as we have too many columns and transformations.

[Nagaraju_Julaka]

@ [ForbinCSD],
We have a requirement to write data from one file to other file based on the mapping provided in the csv file as the number of columns in data file is huge we are generating a mapper snap dynamically as a separate pipeline using the approach suggested in below topic(Using Templates to drive Smartlink or JSON output) and calling the child pipeline in parent pipeline to get the mapper output.

Please suggest if any other better approach to generate mapper dynamically using input csv file.

[ForbinCSD]

Ah. Now it’s becoming a bit more clear what you’re doing and why. Thanks!

Nope, I don’t know right offhand of a better way to handle that.