cancel
Showing results for 
Search instead for 
Did you mean: 

How to remove columns with null value alone in pipeline?

amardeep2021
New Contributor III

Hi,

I have a scenario where I have few columns have null value few not. Objective is to not to bring in columns with null value for each records.

source:

id a b c d e
abc null 1 2 null null
def 11 null 22 33 null
ijk 111 222 null 333 444
lmo null null null null 5555

Expected:
abc 1 2

samething with other records as well. Does anyone know any trick on this? Please let me know.

Regards,
Amar.

6 REPLIES 6

Not sure what you mean. Can you share a version of the input-output example?

marjan_karafilo
Contributor

Hi @amardeep2021
As @cjhoward18 mentioned, you can achieve this by using “filter”.

Here is the input of the file:
image

Expression in mapper:
image

Output:
image

BR,
Marjan