This document demonstrates how to integrate 3rd party job scheduler (e.g. Cisco Tidal) to kickoff and monitor snaplogic pipeline execution.
Deploy the SnapLogic integration pipeline as a trigger task
Invoke the SnapLogic pipeline execution from Tidal through the URL (ground or cloud)
Design the long running pipeline to be asynchronous to avoid timeout (firewall, Snaplogic control plane)
Use the SnapLogic Pipeline Monitoring API to check for execution status
You can find the Snaplogic pipeline monitoring API here
A master pipeline to accept all the Tidal request and distribute accordingly (Triggered)
Call different snaplogic trigger task based on the request.
Log the Tidal Job ID & SnapLogic pipeline runtime ID into Database for status check
Database table to include the key fields: Tidal Job ID, SnapLogic Pipeline Runtime ID, Status
Response to Tidal request to close the connection to avoid time-out
A monitoring pipeline to update Tidal regarding the execution status (Scheduled)
Query the Database for running pipeline
Obtain the pipeline execution status using the SnapLogic pipeline runtime ID (Rest Call) e.g.
Update the Tidal regarding the execution status using Tidal Job ID
Update the Database with the new status