When I selected the option “Reuse executions to process documents” and leave “Pool Size” as 1, it seems that child pipeline is executed the number of documents passed from the previous snap.
This is a portion of my pipeline and mapper snap sends documents about 10000.
This is the configuration I chose on the pipeline execute(My ultimate goal is that child pipeline should run only once and process all the documents received from previous snap).
But we are receiving HTTP status code 429(too many requests in the specified time) in the child pipeline. I believe child pipeline is executed separately for each document sent by mapper snap.
Can someone help on this? I am not sure if I should open a ticket for this.