Routing Error Views

[NAl]

I have a pipeline that has a couple of Salesforce upsert snaps followed by Salesforce Poller ones.
Each of these Salesforce Poller snaps route error data to error view which calls an error pipeline.

When trying to test the error pipeline using a JSON generator snap, the pipeline is completed but there are no json/csv files containing the error records written to the error pipeline. Instead I can see in Salesforce there is a problem with the bulk load and see the issue highlighted in the pipeline statistics.

Here is my error pipeline:

Where should I focus/what steps do I need to take to pass the error records through successfully?

[SpiroTaleski]

@NAl

First, please check “Ignore empty stream” check box in JSON Formatter and CSV Formatter Snap. With that checkbox unchecked, the snap will produce empty result, even on input there will be no data.

Second, the functions the you are using snap.original.load() and snap.original.id can be used only in case of ultra pipelines(If the snaps that contains the functions are inside the ultra pipeline).

So, in order to get the error data, place one file writer or record replay snap inside the error routine, and store the incoming error message. Then using that message you can manually map all the necessary fields.

Regards,
Spiro Taleski

[NAl]

Thanks Spiro, I’ve checked the setting on both Snaps now.

Oh right, didn’t know that. We are using a scheduled task to trigger the pipeline at regular intervals. The data structure changes throughout the pipeline and I’m worried about having to place a route error data to error view option on every snap. Ideally I want to be able to send the error data to the pipeline and be able to map the data once to csv. Is this achievable using a function similar to the one above?

[SpiroTaleski]

@NAl

Yes, if the pipeline is linked with the error routine(pipeline), all the errors that may occurs in every place will be routed to the error pipeline. You can remove all the error views from the Snaps.
The structure of every error message is almost the same. For mapping purposes, you should only store one error message, and use that message for building the mapping in error routine.

Regards,
Spiro Taleski

[SpiroTaleski]

@NAl

The problem on the error routine pipeline. The CSV Formatter Snap is expecting flat data on the input.

Regards,
Spiro Taleski