Error Handler for Validation
Submitted by @stodoroska from Interworks This pattern should be called as an error pipeline and stores all the error records in to a csv file. Configuration Parent pipeline should be made that will invoke this as a error pipeline. Target location is pipeline parameter. No need for changes in the code. Sources: Pipeline that will call this pipeline and will send JSON-formatted data Targets: CSV file on the file sharing system Snaps used: Mapper, CSV Formatter, File Writer Downloads Error Handler for Validation.slp (7.2 KB)
Reference Implementation - Integration Pattern - Error Handling and Retry Pattern
Error Handling and Retry Pattern This pipeline capture errors thrown by error view and invokes an error pipeline. The error pipeline captures the exception encountered and records all the data relating to the execution (Snap level and pipeline level) into a database schema. One could potentially use a database or Kafka or any JMS-compliant queuing mechanism to log the errors. Video Pipelines Attached Error Handler Pipeline_2018_06_25.slp (28.8 KB) Retry Handler_2018_06_25.slp (29.8 KB) The following are error-prone pipelines for use in testing the Error Handler pipeline. Pattern 1 - Step 1.0 - Oracle to Redshift Parent_2018_06_25 (1).slp (8.7 KB) Pattern 1 - Step 1.1 - Oracle to Redshift No Shredding_2018_06_25 (1).slp (24.4 KB) Pattern 1 - Step 1.2 - Oracle to Redshift Shred_2018_06_25 (1).slp (8.3 KB)