Mapper: map to target path only if input path is not null

[mmindenhall]

I’m building a pipeline to update some existing data, and I have a Mapper mapping from source input fields to destination target fields. The input data is incomplete – i.e., there are a lot of fields that can be null on any given input document. For some fields in the input document, if the value is null, I want to ignore it, but if the value is non-null I want to write it to the corresponding target field. The goal here is to avoid overwriting non-null values in the target data with null.

Examples:

Input $first_name == null → ignore, do not write to $First_Name field in target
Input $first_name == 'Bob' → write to $First_Name field in target

It would be really cool if there was a way to configure mappings to ignore null values directly in the Mapper, but I’m not seeing how that can be done. Any suggestions for how to do this (either with something I’m missing in the Mapper or with a combination of other snaps)?

[bojanvelevski]

Try the following expression:

$.filter((v k)=> v!= null)

Map $ as target path.

[darshthakkar]

@mmindenhall: You can also use either of a filter snap or router snap.
When you’re using either of these snaps, you’ll have to define a rule, like $first_name != null

Have a mapper after filter and organize the data in the way you want, write it to a file/update the DB as per your use case.

[cdmills]

This sounds like a job for the Conditional snap; it is essentially “if (condition) then map”.

[bojanvelevski]

@cdmills
The expression I sent is the one you need. Put it in a mapper and validate the result.