Remove unwanted characters from keys or column headings

Designing Pipelines
#1

I have a pipeline which reads zip files which contain csv files and loads the files into sql table.

The zip files contain csv files with column headings which contain spaces, ()/ etc and I would like to remove these fromt he column headings dynamically.

e.g.
Column head 1 => ColumnHead1
Column/head (2) =>. ColumnHead2
etc

I have used $.mapKeys((value, key) => key.toLowerCase().split(’ ‘).map(x=> x.capitalize()).join(’’)), Which removes spaces and Capitalises OK, but I cant figure out how to remove the other characters.

Thanks

#2

Hey @peter,

Use the .replaceAll() function for all of the characters. It will look something like this:

$.mapKeys((value, key) => key.toLowerCase().replaceAll(' ','').replaceAll('(','').replaceAll(')',''))

There’s an option to list all characters in one function, like 'Replace all of these characters with ‘’ ’ but until I found the exact syntax, you can use a separate function for every character you want to remove.

Bojan

2 Likes