With an Ultra Task, there are multiple pipeline executions (i.e. instances) that are used to process requests sent to the task. Each request is turned into a document that is fed into one of the executions for processing. Since all of the snaps in a pipeline run in parallel, a pipeline can process many documents simultaneously. Documents that are being processed by a pipeline are “in-flight”. So, the ‘Max in-flight’ parameter restricts how many documents/requests are allowed to be processed by an execution before it will stop accepting more.
Why does this matter? If a pipeline calls out to external services, one of the executions may be slower than the others and that delay will compound for subsequent requests. If this is possible, lowering the ‘Max in-flight’ allows subsequent requests to be processed by other executions that may be faster. For example, if an execution is experiencing a 10 second delay when communicating with a service and 4 requests are fed into this execution with no delay, the first request will get a response after 10 seconds, the second 20 will be seconds later, and so on. However, if the ‘Max in-flight’ was lowered to 1, the second and third requests could be sent to other executions that may be performing better.
What’s the downside to lowering this parameter? It will slow down the performance of the happy path since it disables prefetching requests from the feed-master.
What’s the downside to lowering this parameter? It will slow down the performance of the happy path since it disables prefetching requests from the feed-master.
Can you elaborate this statement
I did not follow it
What Tim Stack was referring to here is the case where the pipeline is not bottlenecked by an external service or application. In that case, it’s preferable to set max inflight to a higher value in order to reduce the back and fourth between the pipeline and the FeedMaster. For example, if there are 10 documents to process and max inflight is set to 10 (or higher), the FeedMaster will push all 10 documents to the pipeline in one go. Whereas, if the max inflight was set to 1, then each document would need to be processed individually before another would be fetched requiring 10 individual requests.
