Data Validation using Data Catalog

#1

Hi all,

I’m trying to do some data validation on each field of data in each column.
I have the validation parameters for each column heading, in a row on a separate table (Azure Data Catalog).

Example:
Data - (table name)
FirstName - (column)
John
Mary

Validation - (table name)
column_name | data_type | length - (columns)
FirstName | text | 20

So I would like to validate the values of “FirstName” against the validation parameters in the Validation table.

Any thoughts on this? Or if anything needs more clarification?

#2

@philliperamos

Can you please provide more clarification what you want to validate?

#3

Hi Supratim,
I figured out a way to get this done.
I converted the data to a JSON format to pivot the data, then I used a combination of JSON Splitter/Mapper and Data Validator to validate the data.

#4

Well done phillp.