โ06-24-2022 11:12 AM
Iโm building a pipeline to update some existing data, and I have a Mapper mapping from source input fields to destination target fields. The input data is incomplete โ i.e., there are a lot of fields that can be null
on any given input document. For some fields in the input document, if the value is null
, I want to ignore it, but if the value is non-null I want to write it to the corresponding target field. The goal here is to avoid overwriting non-null values in the target data with null
.
Examples:
Input $first_name == null
โ ignore, do not write to $First_Name
field in target
Input $first_name == 'Bob'
โ write to $First_Name
field in target
It would be really cool if there was a way to configure mappings to ignore null values directly in the Mapper, but Iโm not seeing how that can be done. Any suggestions for how to do this (either with something Iโm missing in the Mapper or with a combination of other snaps)?
โ06-25-2022 12:02 AM
Try the following expression:
$.filter((v k)=> v!= null)
Map $ as target path.
โ06-25-2022 08:18 PM
@mmindenhall: You can also use either of a filter
snap or router
snap.
When youโre using either of these snaps, youโll have to define a rule, like $first_name != null
Have a mapper after filter and organize the data in the way you want, write it to a file/update the DB as per your use case.
โ06-29-2022 01:13 PM
This sounds like a job for the Conditional snap; it is essentially โif (condition) then mapโ.
โ06-30-2022 12:32 AM
@cdmills
The expression I sent is the one you need. Put it in a mapper and validate the result.