Error Pipeline Feature

[parvathy_r]

This is regarding the 2017 Error Pipeline Feature introduced in Fall Release 4.11 which is now available in UAT.

This feature would help us in making the pipeline design look neater by eliminating the explicit exception handlers from the pipeline design. But I was wondering how to handle the below mentioned scenario:

The exception handler details is given at the pipeline level. But if I want to capture the exception in each snap into separate file (Assuming that my exception handler would look like JSON Formatter => File Writer), I will parameterize the ErrorFileName. But this should look like ‘Err_’+snap.label+‘.json’. But since we are giving this expression at the pipeline level, snap.label doesn’t make sense. How can I generalize the same?

[tstack]

parvathy.r:

The exception handler details is given at the pipeline level. But if I want to capture the exception in each snap into separate file (Assuming that my exception handler would look like JSON Formatter => File Writer), I will parameterize the ErrorFileName. But this should look like ‘Err_’+snap.label+’.json’. But since we are giving this expression at the pipeline level, snap.label doesn’t make sense. How can I generalize the same?

The error documents created by most snaps include a ‘snap_details’ property that contains the label of the snap. So, you can build the error pipeline to work with that property. As for writing to separate files, the JSON Formatter snap now has the ability to insert document data into the header of the binary output document. The File Writer can then be configured to create the output file name based on values in the header. I’m attaching an example project to demonstrate this flow, it contains a Main pipeline with some errors that are sent to the ErrorHandler pipeline. Note that creating separate files for every error document will be fairly slow, so it might work better to create a single file.

ErrorPipelineDemo.zip (2.9 KB)