Error Handling and Retry Pattern
This pipeline capture errors thrown by error view and invokes an error pipeline. The error pipeline captures the exception encountered and records all the data relating to the execution (Snap level and pipeline level) into a database schema. One could potentially use a database or Kafka or any JMS-compliant queuing mechanism to log the errors.
Video
Pipelines Attached
Error Handler Pipeline_2018_06_25.slp (28.8 KB)
Retry Handler_2018_06_25.slp (29.8 KB)
The following are error-prone pipelines for use in testing the Error Handler pipeline.
Pattern 1 - Step 1.0 - Oracle to Redshift Parent_2018_06_25 (1).slp (8.7 KB)
Pattern 1 - Step 1.1 - Oracle to Redshift No Shredding_2018_06_25 (1).slp (24.4 KB)
Pattern 1 - Step 1.2 - Oracle to Redshift Shred_2018_06_25 (1).slp (8.3 KB)
