Forum Discussion

philliperamos's avatar
philliperamos
Contributor
6 years ago

Data Validation using Data Catalog

Hi all,

I’m trying to do some data validation on each field of data in each column.
I have the validation parameters for each column heading, in a row on a separate table (Azure Data Catalog).

Example:
Data - (table name)
FirstName - (column)
John
Mary

Validation - (table name)
column_name | data_type | length - (columns)
FirstName | text | 20

So I would like to validate the values of “FirstName” against the validation parameters in the Validation table.

Any thoughts on this? Or if anything needs more clarification?

3 Replies

    • philliperamos's avatar
      philliperamos
      Contributor
      6 years ago

      Hi Supratim,
      I figured out a way to get this done.
      I converted the data to a JSON format to pivot the data, then I used a combination of JSON Splitter/Mapper and Data Validator to validate the data.