Forum Discussion
ptaylor
5 years agoEmployee
@acesario I think what I’m understanding is that one of your files might have an array with more than one element, but the default parsing turns each element into a separate output document. You can change that by setting “Process array” to false on the JSON Parser. That will produce a single output document whose data is the full array. Before you can feed that into the Join, you’ll have to use a Mapper to map the array to the field of an object. I’ve attached a sample pipeline.
Community 8107_2020_08_27.slp (9.8 KB)
Here are the inputs…
fileA.json:
[
{
"file": "A"
}
]
fileB.json:
[
{
"file": "B",
"id": 1
},
{
"file": "B",
"id": 2
}
]
Output of the Join:
[
{
"fileA": {
"file": "A"
},
"fileB": [
{
"file": "B",
"id": 1
},
{
"file": "B",
"id": 2
}
]
}
]
That combines all of the data from both input files into a single document. You can modify the Mappers to move things to the right places.