Forum Discussion

jpanikulam's avatar
jpanikulam
New Contributor II
8 years ago

Throttle Pipeline based on a combination threshold

Hi,

We have a need to trigger a child pipeline when an input SQS message count reaches ‘N’ messages OR ‘M’ seconds have passed since the last run - whichever occurs earlier. Any thoughts on how best to write this pipeline?

The goal is to try and throttle these fast incoming messages and process in groups rather than one by one since otherwise can result in a lot of processing and network traffic.

Thanks,
Joseph

2 Replies

  • aleung's avatar
    aleung
    Contributor III

    This probably need a bit more details to resolve. Feel free to reach out to me.