Forum Discussion

rpatrick00's avatar
rpatrick00
Contributor
5 years ago

Mapping optional arrays of objects to arrays of objects

I have an input schema that contains some optional arrays of objects in one format that I need to map to a target schema with an array of objects in another format.

The problem I am having is how to get the Mapper to map the array if it is present and ignore it if it is not. For example, my mapping table expression for one field below works fine for documents where the Address element is present but causes an error if it is missing.

jsonPath($, "$MatchUpdatePartyPayload.Address[*].NonStandardUS.Line1") 

Thanks,
Robert

12 Replies

  • Checking the ‘Null-safe access’ property in the mapper will allow for accessing a null field and will give you a null value instead, which can be dealt with downstream possibly?

  • While that gets rid of the error, it just creates more problems downstream with my custom snap’s validation code. Is there some other way to accomplish this sort of mapping?

    • cjhoward18's avatar
      cjhoward18
      Employee

      What would you like the value to be when it is not present?

  • I would like the mapper to simply ignore the entry and move to the next one…

    If I have 10 lines in my mapping table and the input field(s) are not present for 5 of those lines, then the mapper should only map the 5 lines that it has input data for…

    • cjhoward18's avatar
      cjhoward18
      Employee

      And if there are none then you’d want an empty array ?

    • cjhoward18's avatar
      cjhoward18
      Employee

      The reason I ask is I am assuming these are all separate documents coming in to the mapper that have a field $MatchUpdatePartyPayload, which may or may not contain an array at the Address key.

      For instances in which a document does not contain the specified array, what would you expect the expression to evaluate to in that instance?

      It is not possible to simply skip the evaluation/mapping of that expression based on that key not being present. Some value needs to be used for the output of the expression and map to the target field. If you can group your documents, you could filter them based on the presence of this array and extract all values where applicable. Hope this helps in some way.

      • rpatrick00's avatar
        rpatrick00
        Contributor

        The actual payload is a little more complex than that.

        The payload really looks like this (even this is somewhat simplified):

        {
            "AddAttorneyStandingPayload": {
                "AttorneyID": <integer>,
                "CurrentStanding": <string>,
                "Date": <string>
            },
            "MatchUpdatePartyPayload": {
                "PartyID": <integer>,
                "Name": {
                    "Title": <string>,
                    "First": <string>,
                    "Middle": <string>,
                    "Last": <string>,
                    "Suffix": <string>
                },
                "Address": [
                    {
                        "NonStandardUS": {
                            "Line1": <string>,
                            "Line2": <string>,
                            "Line3": <string>,
                            "City": <string>,
                            "State": <string>,
                            "Zip": <string>
                        },
                        "Foreign": {
                            "Line1": <string>,
                            "Line2": <string>,
                            "Line3": <string>,
                            "Line4": <string>
                        }
                    }
                ],
                "Phones": {
                    "Phone": [
                        {
                            "Number": <string>,
                            "Extension": <string>,
                            "Type": <string>
                        }
                    ]
                },
                "Emails": {
                    "Email": [
                        {
                            "EmailAddress": <string>,
                            "IsCurrent": <boolean>
                        }
                    ]
                }
            }
        }
        

        In this payload, the MatchPartyUpdatePayload element itself is optional (if the name, address, phone, and emails hasn’t changed, no need to specify the payload). Within the MatchPartyUpdatePayload element, Name, Address, Phones, and Emails are all optional (they are only needed if the data is being changed). Within some of those arrays, the objects themselves have optional fields (e.g., Line2 and Line3 for the NonStandardUS object).

        The problem with the null-safe access is that it is putting the null at the lowest level. For example, let say that I have no Address element, what I am ending up with is a fully populated Address element with all values set to null. This doesn’t work for me because the NonStandardUS address element, if present, has 4 required fields and 2 optional fields. And the Foreign and NonStandardUS fields are mutually exclusive. My validation code verifies that any Address element has at least the 4 fields set to non-null/non-empty values. What would be better for me would be for the entire address list to be either empty or set to null without creating the nested address object and setting all 6 fields to null.

        My company has some very large payloads (100s of fields with optional elements at various points in the message hierarchies, some of which are objects or arrays of objects). Having the Mapper set dozens of leaf node fields to null quickly becomes intractable since I would have to write a script to walk the entire tree and prune off the elements where all fields are set to null, starting at the leaf nodes and working my way back up. If I have to do that, I might as well just write a script to do the mapping the way I need it in the first place, right?

        Hope this helps,
        Robert

  • If you enable the Null-safe evaluation, then you could follow this Mapper with a second Mapper to prune out the null or blank fields with a mapping like this:

    Expression:    $.filter((value, key) => value && value.trim())
    Target path: $
    

    See the Preview panels in this example:

  • Agreed. I had thought about the custom function in the expression language. The only issue I have with that one is that it is also, if ever so slightly, non-intuitive that you change behavior of the entire mapping operation for that row with a function in the expression (which typically just manipulates what data gets assigned).

    If you really wanted to be able to set different policies for each row in the mapping table, you could add some sort of per-row setting which would clutter up the UI but might be a little more intuitive. For example, you could set the behavior at the mapper level and then override a particular row with the per-row setting…

    Anyway, I will leave it to you to decide how you want to surface the functionality. I am just happy that you see the need for this.