04-02-2024 05:35 AM
I Want to know the all the input columns in the pipepline using expression in mapper snap. I used $.keys() function in mapper snap its gives information for all the rows(if we have 1lakh records for processing, its gives information for all 1 lakh records...it leads to performance issue). I need only 1 row output which gives all the column names.
I don't use head,filter snap for this task which makes another line in the pipeline ... to eliminate that we want to write expression in mapper snap.
04-06-2024 10:20 PM
Have you tried batching the input like batch of 1000 or 10000 and then process the data??