Dynamic Data Validation
I want to validate the incoming data which will be in CSV format. I have Target table defined for each file. Now I want to develop a common pipeline which should validate the data based on the rule f...
Forum Discussion
the way we developed, we don’t need to make changes to the snaplogic pipeline if we want to validate different set of files. it could be new set of validation or few new column validation to the same set of files. all we need to do is add validation rule to the table and corresponding file name. its more dynamic validation of data and column name in a file
Thanks. We’re thinking of doing something similar for the dynamic validation approach, as the way I described above is leading to huge memory problems.
Hi @munish,
Do you know the number of sheets in the Excel file, and their respective names, in advance?
If you do know this information in advance, it could be relatively easy to accomplish this by using a child pipeline in which you can implement you logic for reading from the Excel file and writing the data to a JSON file.
Cheers,
Dimitri
