Issue with Arrays in Downstream Mappers after a Custom Snap Transformation

[cclaudio]

We built a custom snap that performs a transformation. You will notice in the output of the custom snap in the screenshot below that “medicalEnrollment” is a json array.

In a downstream mapper, we attempted a transformation on this array. We have tried (1) using the sl.ensureArray() method and (2) another downstream mapper performing a transformation in the array. However, we notice that in both the #1 and #2 attempts the mappers do not detect the array, thus causing issues with the transformation.

Below is a snippet of the preview of what we expect vs actual preview.

We also notice that chaining a Json Formatter and a Json Parser right after the custom snap does indeed makes the mappers work correctly. Pattern shown below:

So my question is, is there something hidden in the JSON parser snap that allows strong typing of the array that we are missing? Anything in between?

@robin , Tagging you since I was told you might be able to help out. Thanks!

[ptaylor]

cclaudio:

We are using a HashSet. We will try converting to ArrayList and post an update tomorrow.

I think this is the issue. If you want this object to be treated as a JSON array, its class needs to implement Java’s List interface. HashSet implements Collection, but not List, so you’ll get a weird mix of behaviors from the subsequent processing, as some of it will work fine with any Collection, but some aspects like the Mapper will only recognize a List as an array.

[cclaudio]

@ptaylor, We were not able to make it work by switching to ArrayList. We will create another custom snap simplified to bare bones and use an ArrayList. Are you aware of any other requirements that the Mapper might need to interpret the Arrays correctly?

[ptaylor]

That’s unfortunate. I am not. This will be difficult to diagnose without a way for us to reproduce the issue. Screenshots and text descriptions won’t allow us to debug. Is there any way you can help us to reproduce this problem?

[ptaylor]

I suggest creating a support ticket and providing us with a link to a test pipeline that we can execute.

[cclaudio]

I suggest creating a support ticket and providing us with a link to a test pipeline that we can execute.

@ptaylor I already have created a ticket with support (Ticket #43346). I will follow up there with the details.