Depending on your configuration, you might be able to have one primary triggered task that fire off the others. We have a case where I have one snaplogic REST API that takes in parameters to invoke other pipelines ( > 20). So I don’t have 20 different REST URL’s to provide - just one and then let the parameters determine what should be executed next. If you’re having 20/sec, the snaplex you set up should have multiple nodes to handle the traffic. Each snaplex can handle 10 concurrent processes last I heard. So you would need to have multiple nodes for that snaplex to distribute the work.
Very interesting! Also, if you have a link to the documentation specifying that a Snaplex can handle 10 concurrent processes, could you share that data? I would like to learn more about that.
@chenry, we were told that as best practice from the PS team I believe. I don’t have any written documentation on that part.
Hi Everyone,
This is an interesting topic and it would be great if we can get some kind of documentation on the limitation of using trigger tasks.
In my project we are also trying to do something similar. Our use case is we are getting data from Kinesis stream and since currently Snaplogic doesn’t have any snap for it, we are thinking to use Snaplogic trigger task URL as a post request from lambda function. So, whenever our stream gets data the lambda will execute and it will send that data to Snaplogic pipeline using the post request. We did a test with sample stream and it worked fine. However, we are not sure how it is going to perform with actual stream data with lots of data coming every minute. It would be great if get any kind of documentation on trigger task limitation so that we can take care of it in our implementation.
Thanks
