JSON Splitter for an Split column

[pranil2k2]

Hi,
I have a requirement to read multiple XML files data and load into Oracle table. I am using Directory browser to read the list of XML files and passing the path to File reader and then XML Parser, from XML parser the output is given to mapper there i am splitting the data on a column using xx.split(‘\n’). The output from mapper is given to JSON Splitter to split on xxx, this is working good for one XML file, if i try to process for multiple XML files the JSON split is not working as excepted.

Even i tried by creating child pipeline and passing path value, but i am getting same results. it is splitting one XML properly and other XML with only one record.

Please some one can advice here.

As per my understanding, the Directory browser will execute one file at a time, please correct me if i am wrong.

[SpiroTaleski]

Hi @pranil2k2

The Directory Browser lists all the files from the specific location based on the given criteria, and then the File Reader will process one file at a time.

One possible reason why JSON Splitter is working only for one file it can be that the other XML files does not have the same structure as that file(which the JSON Splitter works properly for), or that “xx”(split) column does not exist in the other files, or it may be empty string or null.

You should check/handle those scenarios before the splitting logic 🙂 .

Regards,
Spiro Taleski

[pranil2k2]

HI Taleski,

Thanks for the reply. to avoid those confusions/checks i have taken only two files that have complete data. i have executed the files individually and it is getting all results as shown below. Here the file list is stored in excel and act as input to directory browser and when trying to execute two XMLs the JSON splitter is not splitting as excepted, it is reading only one record from first XML. This is where i need advice… is reading file names from excel is not correct one? is something wrong here. please advice.
XML one

XML two

Reading two XMLs.

[pranil2k2]

Please some one advice here…

Thanks

[ptaylor]

There’s an inherent ambiguity with converting XML to a JSON-like structure with regard to knowing which elements should be converted to an array. When there are multiple adjacent elements with the same name, that’s obviously an array. But in the case where there’s only one element with that name, should that be an array with one element, or just an object?

The key to solving this issue is to call the sl.ensureArray function in the Mapper right before the JSON Splitter. Map the path you’re using for the split expression to sl.ensureArray(path). Use that when you know that the path is supposed to represent an array. When the XML has more than one element for that path, the value passed to that function will already be an array, and the function will just return that array without modification. But when the output of the XML parser has a single object at that path, the function will wrap that object in an array and return that. So it will work in both the single- and multiple-element cases, because it ensures that the JSON Splitter will always operate on an array.