How to remove columns with null value alone in pipeline?

[amardeep2021]

Hi,

I have a scenario where I have few columns have null value few not. Objective is to not to bring in columns with null value for each records.

source:

id	a	b	c	d	e
abc	null	1	2	null	null
def	11	null	22	33	null
ijk	111	222	null	333	444
lmo	null	null	null	null	5555

Expected:
abc 1 2

samething with other records as well. Does anyone know any trick on this? Please let me know.

Regards,
Amar.

[cjhoward18]

Not sure what you mean. Can you share a version of the input-output example?

[marjan_karafilo]

Hi @amardeep2021
As @cjhoward18 mentioned, you can achieve this by using “filter”.

Here is the input of the file:

Expression in mapper:

Output:

BR,
Marjan