Confused with Pipeline execute options

[ddangler]

When I selected the option “Reuse executions to process documents” and leave “Pool Size” as 1, it seems that child pipeline is executed the number of documents passed from the previous snap.

This is a portion of my pipeline and mapper snap sends documents about 10000.

This is the configuration I chose on the pipeline execute(My ultimate goal is that child pipeline should run only once and process all the documents received from previous snap).

But we are receiving HTTP status code 429(too many requests in the specified time) in the child pipeline. I believe child pipeline is executed separately for each document sent by mapper snap.

Can someone help on this? I am not sure if I should open a ticket for this.

[tstack · Accepted solution]

ddangler:

My ultimate goal is that child pipeline should run only once and process all the documents received from previous snap

The configuration shown is correct for what you want here.

ddangler:

I believe child pipeline is executed separately for each document sent by mapper snap.

Check the SnapLogic Dashboard to see how many child executions there actually are. There should only be one.