I am trying to build a pipeline where a series of format operations must be applied to a CSV file. For some columns, I want to delete/remove or simply ignore empty rows from the file. I tried doing this with a mapper/filter but with no success.
What will be the best way of doing this?
Use a mapper with an object filter function. Check below:
$.filter((v,k)=> v != null)
Note that this expression will work on a flat Json structure only.
Hope this helps,
Thank you @bojanvelevski! Can you do this via a filter snap as well on a column?
@AlexVlad , the Filter snap has Object oriented functionality. Filters out incoming objects based on a certain condition.
The expression I sent is oriented on the structure on each of the incoming objects. Meaning, it will filter out specific fields of the Object, based on a certain condition.