07-06-2024 03:58 PM
I am building a generic error pipeline which will load error occuring in the source pipeline and write them/load them into a database. It will load data such as label, user, etc. on which the error takes place.
In source pipeline, within within the pipeline parameters, I have used "Error Pipeline" feature to input the pipeline that would handle error and load the data. But after defining the "Error Pipeline Parameters" (such as pipe.label), which will act as output from the source pipeline (and input to generic pipeline), the mapper within the generic pipeline fails to receive/identify parameters from source.
I have used _ in the mapper expression followed by key name to call them into generic pipeline, but cannot do so. Is there any other way ?
This is the source pipeline
This is my error generic pipeline
Solved! Go to Solution.
07-08-2024 08:20 AM
Every parameter that you will define in the Error Pipeline Parameters on the main pipeline will be propagated to the Error pipeline(in case of error).
If you want to do a testing/validate only on error pipeline, you should define Parameters on the Error pipeline so you will not get the error as previous:
07-09-2024 12:52 AM
The in.TotalCound and out.totalCount functions will returns total number of documents that have passed through all of the Snap's.
About the resource utilization metrics, check the following post/pattern that uses the APIs for getting the current resource utilization.
07-09-2024 06:54 AM
I tried out in.TotalCount within the pipeline parameters but receive an error
Error is as follows, takes place while validating the pipeline. I guess it not loading from parent pipeline to error handler
07-09-2024 07:13 AM - edited 07-09-2024 07:14 AM
Unfortunately that function cannot be evaluated there/in pipeline parameters.
Both functions, in.TotalCound and out.totalCount gives the total input/output documents passed through the snap(where the functions are implemented).