Remove multiple columns from writing into a file

Designing Pipelines
1

i’m looking for an approach where i can remove multiple columns from writing into a file. The catch here is I don’t want to hardcode the column names (at source column) and passing null (in target column).
I want to identify the unwanted column based on the prefix (say Join_) and remove all those columns from writing into a file.

Say Im getting column
A
B
C
D
Join_A
Join_B

I don’t want to write Join_A and Join_B in file. I can’t hardcode the column names because I’m not sure how many such columns with that prefix i will get.

Any pointers would be appreciated. tia

2

Welcome to Community @gunja

I believe the attached pipeline will provide an example of what you’re trying to do.
Community 15779 - Remove columns_2023_03_02.slp (4.9 KB)

The expression in the Mapper snap uses the Object.filter() method to determine the field names to remove. This does assume that the field names to be removed are at the root of the object.

I hope this helps!