How to read the file only once and perform multiple validations on the file.
My file has 20 columns and I want to find the count of 4 different columns (Column1, Column4, Column9 and Column19).
I have the logic to get the count built in a child pipeline and the Parent pipeline passes the column names for which the count has to be found.
Since the columns can be changing for each file, we have made this dynamic.
Issue here is, we are reading the file in teh child pipeline and based on teh column names passed from the parent pipeline, we are getting the count.
If we need to get the count for 4 columns, the child pipeline will execute 4 times and the input file is being read 4 times. Is there a way we can avoid reading the file multiple times.
Example: I want to find the count in Message, Invoice_id and amount columns.