JSON returns Column Names separately from the Rows

Designing Pipelines
#1

I have a REST API that returns something like the following:

entity":
{
“items”:
[
{
“tableName”:
“mytable”
“count”:
7995
“columnNames”:
[
“c1”,“c2”,“c3”,“c4”
]
“rows”:
[
[r1c1value,r1c2value,r1c3value,r1c4value],[r2c1value,r2c2value,r2c3value,r2c4value],[…],[…],[…],[…],[…],[…],[…],[…]
]
}
]
}

When I refer to jsonPath($, “entity.items[*]”) it gives me columns as an array of columns and the rows are an array of rows.

How can I parse this so that I can map it to a Mapper to put into SQL?

Thanks

#2

Hi @Garrett

Probably, you should try with JSON Splitter Snap, and try to split the arrays containing the values/fields that you want to map further.

Regards,
Spiro Taleski

#3

pull out two arrays separately using a mapper 1) entity.items.columnNames 2) entiy.items.rows and then map the values.

#4

@Garrett,

If I captured your input data correctly, this attached pipeline should help get you started:

Community.9525.slp (4.8 KB)

The mapper combines the Array map() method with the sl.zipObject() method to bring the column names and row values together in an object array. The following splitter will flatten the data for your SQL Insert.

1 Like
#5

This looks really good and looks like the solution. I am getting an error and it is probably because I’m slightly off on the object above.

Failure: Expecting object for JSON-Path field reference ‘entity’, found: null, Reason: Expecting object for JSON-Path field reference ‘entity’, found: null, Resolution: Change the path to refer to an object

It really is:
{statusline:{},entity{},headers{}}

When I refer to anything in there though I have always used $entity as you did so I am not understanding why it isn’t doing the reference properly.

I added a first splitter to get:
jsonPath($, “entity.items[*]”) this then let:

the Mapper Object see columnNames and rows in the Input Schema on the left which when I just connect the JSON output to the mapper there was nothing showing in the Input Schema.

It looks good now but the Json Splitter isn’t returning anything and hence the mapper. No errors. No output.

#6

I got it.

I had to do the splitter first and get [rows] and [columns]
then I was able to modify the sl.ZipObject slightly from your original to:
jsonPath(, "rows[*]").map(x=>sl.zipObject(jsonPath(, “columnNames[*]”),x))

It was like it couldn’t reference it from the entity level.