04-25-2018 06:37 AM
We have requirement where we have multiple data quality rules configured in a Oracle table for each pipeline that loads different tables. Each rule also has a severity associated with it. Ex: If when are reading Employee data we want to check if Employee number is null(High Severity issue) and also Department Code is null(Low Severity Issue). Questions
How do we make these null-ability checks or other generic checks reusable so that we don’t
need to re-code it every single time
Can we make a lookup reusable so that in the same pipeline we can lookup on our rue table and fetch the severity for different rules. So if we have 5 rules, want to lookup 5 times
09-06-2019 08:12 AM
Were you able to resolve this? If so, can you please share the solution?
12-06-2020 03:23 AM
Hi,
We have a similar scenario. Can you please share the solution if it is resolved.
12-07-2020 04:06 PM
One way to do this could be writing the Expression(Null Check or any validation Check) in the .expr file and then do a lookup.i.e lib.conf.validityCheck($Event.id) ( Field Name $Event.id you want to perform the validation on)
Refer to the documentation for more details.
https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1438110/Expression+Libraries