Create reusable lookup that we call multiple times in the same pipeline
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
‎04-25-2018 06:37 AM
We have requirement where we have multiple data quality rules configured in a Oracle table for each pipeline that loads different tables. Each rule also has a severity associated with it. Ex: If when are reading Employee data we want to check if Employee number is null(High Severity issue) and also Department Code is null(Low Severity Issue). Questions
-
How do we make these null-ability checks or other generic checks reusable so that we don’t
need to re-code it every single time -
Can we make a lookup reusable so that in the same pipeline we can lookup on our rue table and fetch the severity for different rules. So if we have 5 rules, want to lookup 5 times
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
‎09-06-2019 08:12 AM
Were you able to resolve this? If so, can you please share the solution?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
‎12-06-2020 03:23 AM
Hi,
We have a similar scenario. Can you please share the solution if it is resolved.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
‎12-07-2020 04:06 PM
One way to do this could be writing the Expression(Null Check or any validation Check) in the .expr file and then do a lookup.i.e lib.conf.validityCheck($Event.id) ( Field Name $Event.id you want to perform the validation on)
Refer to the documentation for more details.
https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1438110/Expression+Libraries
