We’ve been discussing the idea of making sample pipelines available in the Community. While the specifics of how this would happen still needs to be worked out, we’d liked to know what kind of use cases would you like to see?
How to use and interrupt the Pipeline and Snap monitoring APIs to uncover bottlenecks. Including Pipelines with child Pipelines
How to do date conversions from RDBMS Datetime types using the Date functions. All the examples start with Date.now() which output snaplogic compatible types which aren’t as helpful when dealing with RDBMS types.
I would love to see a whole category on performance monitoring and tuning tips. Environment, Pipelines, Snaps and expressions.
I would love to see a whole category on performance monitoring and tuning tips. Environment, Pipelines, Snaps and expressions.
Do you mean a category within the Community for Performance? If so, that would be Monitoring Pipelines and Plexes.
Is there a way to figure out which snap in a pipeline is causing the performance bottleneck ? Not sure how accurate the statistics information of snap in pipeline execution statistics .
Have you gone through the documentation for the execution statistics dialog? In particular, pay attention to the Duration bars for the Snaps. Snaps with long blue bars are spending their time executing, while Snaps with long green and purple bars are waiting for upstream or downstream snaps to finish working. So, snaps with long blue bars are typically the bottleneck.
Do you have reason to believe they are not accurate? If you can provide some examples, we can take a look.
Not only more time consumed also other factors like memory & CPU usage . Also I meant to do this in a pipeline and send an alert about that pipeline, snaps and other required details. When multiple team using the tool and all of them sharing same nodes for execution, this would be really helpful.
Ultra Pipelines to illustrate best practices on creating Restful APIs to support GET, PUT, POST and DELETE HTTP verbs using a simple CRUD commands on a RDBMS structure.
