Random behavior from pipeline when trying to work/split on multi-level json data
Case/scenrio: Used assets Sample input xml below multiplemstercpttoub454.txt (51.1 KB) Problematic pipeline asset below but_why.slp (67.8 KB) (you may want to remove the splunk, producer etc snaps if connection is unavailable during investigation) Requirement context around which the problematic random behavior identified: In input xml, following two tags may repeat any number of times: RCPT_INVOICE_OUB_IFD RCPT_INVOICE_LINE_OUB_IFD Now the need was to split this xml into individual xml containing one of each from above tags… then transform those xml into a plain text for output Logic was to ensure both of these are treated as arrays (so even when there is just one value for repeating tags, we would still be able to loop through them after transforming them as array else the same expression may not work for both arrays and map) and split them in order keeping the parent values intact Problem: Random behavior… some times it gives correct 2 outputs where expqty != rcvqty but sometimes… same message is repeated twice!?. Screenshot highlighting areas of interest: Working run details (when the behavior is correct per expectations): This time the two outputs are per expectation: Check the output from 3 So there were two records in input with expqty != rcvqty and those data were correctly split and processed. Note the outputs at 1 Note the outputs at 2 The input and output preview at 2 Now tried re-validating a few times until the random wrong behavior kicks in… Incorrect run details (when the result is wrong) This time the two outputs are wrong. Same message got processed twice: Check the output from 3 So there were two records in input with expqty != rcvqty and one of the records gets repeated… Note the outputs at 1 The behavior is correct till here. It messes at 2 Note the outputs at 2 Input and output preview at 2 How did the value of this tag get swapped? And this happens randomly Extra notes a. I tried different approaches at the point 1 like using mapper, structure etc but the random behavior still persisted. b. As a last resort I had to handle all splitting operation (all of it) within script so the output is always correct… but I am still interested to know the reason for this behavior and a better solution if there is one
JSON Splitter to REST Call capture all response bodies
I have a pipeline that takes an Array of JSON as input. We then feed it to a JSON Splitter that then goes into a REST POST to some endpoint. I see that all of the REST calls go through in the Ultra task, however only the first response body from the first REST call is returned by Snap. Does anyone know how to capture all the response bodies into an array, and only after all REST calls are processed, then SnapLogic sends back its array of responses? Or is this not possible…Alternate for JSON Splitter - Split single document into multiple rows
Hi, I am currently using Json Splitter in a pipeline to split records and send it to Oracle merge. As there are huge records, json splitter processing time is very high. Tried moving the json splitter and Oracle merge snap into a different pipeline and enabled re-use in pipeline execute. Still no improvement in the performance. Kindly let me know if there is any alternate for Json Splitter. Tried the below approach in mapper to split single document (consisting multiple objects) into multiple rows. But it returns array of array. Input file : SampleInput.txt (267 Bytes) Mapper expression: {}.extend($response.map((elem, index) => elem)).values() Target: $ Expected output: Best Regards, PoojaAfter using a JSON splitter and a filter, how do you put the object/array back together?
Hi, we needed to filter on a JSON input. We managed to get that working by using a JSON splitter on the element and then piping that to a filter snap. Our issue is now the input that looked like this: "allLocations": { "location": [ { "addressInternalid": 2631363, "isDelete": false, "internalSupplierid": 3423589, "acctGrpid": "RT", "address1": "5309 GREENWAY", "address2": "5301 REDWAY", "address3": "5504 BLUEWAY", "poBox": "0912KHJWD", "country": "USA", "state": "US-TX", "city": "FREE", "zip": "78211", "phone": "2229808888", "phoneExtn": "091", "fax": "747", "faxExtn": "737" }, { "addressInternalid": 2631367, "isDelete": false, "internalSupplierid": 3423589, "acctGrpid": "RT", "address1": "11305 4 PTS DR", "address2": "BLDG 2,#100", "country": "USA", "state": "US-TX", "city": "AUSTIN", "zip": "78726", "phone": "5126648805", "phoneExtn": "123", "fax": "123", "faxExtn": "134" }, { "addressInternalid": 2631368, "isDelete": false, "internalSupplierid": 3423589, "acctGrpid": "RT", "address1": "REMIT 11305 4 PTS DR", "address2": "BLDG 3", "country": "USA", "state": "US-TX", "city": "AUSTIN", "zip": "78725", "phone": "5126600000", "phoneExtn": "678", "fax": "678", "faxExtn": "678" } ] }, Looks like this: [ { "addressInternalid": 2631363, "isDelete": false, "internalSupplierid": 3423589, "acctGrpid": "RT", "address1": "5309 GREENWAY", "address2": "5301 REDWAY", "address3": "5504 BLUEWAY", "poBox": "0912KHJWD", "country": "USA", "state": "US-TX", "city": "FREE", "zip": "78211", "phone": "2229808888", "phoneExtn": "091", "fax": "747", "faxExtn": "737", "fullCompanyName": "SUPPLIER MARCH 3 dba TEXT", "requestId": 5272423, "id": "3423589", "facilityCode": "0001", "systemCode": "1", "supplierType": "Operational", "status": "ACTIVE" }, { "addressInternalid": 2631367, "isDelete": false, "internalSupplierid": 3423589, "acctGrpid": "RT", "address1": "11305 4 PTS DR", "address2": "BLDG 2,#100", "country": "USA", "state": "US-TX", "city": "AUSTIN", "zip": "78726", "phone": "5126648805", "phoneExtn": "123", "fax": "123", "faxExtn": "134", "fullCompanyName": "SUPPLIER MARCH 3 dba TEXT", "requestId": 5272423, "id": "3423589", "facilityCode": "0001", "systemCode": "1", "supplierType": "Operational", "status": "ACTIVE" }, { "addressInternalid": 2631368, "isDelete": false, "internalSupplierid": 3423589, "acctGrpid": "RT", "address1": "REMIT 11305 4 PTS DR", "address2": "BLDG 3", "country": "USA", "state": "US-TX", "city": "AUSTIN", "zip": "78725", "phone": "5126600000", "phoneExtn": "678", "fax": "678", "faxExtn": "678", "fullCompanyName": "SUPPLIER MARCH 3 dba TEXT", "requestId": 5272423, "id": "3423589", "facilityCode": "0001", "systemCode": "1", "supplierType": "Operational", "status": "ACTIVE" } ] How do I get it back into an object so I can map it correctly? As you can see in the below image, all of the location fields are coming in as strings with no hierarchy, not the array they were originally. Any tips would be appreciated!
How to flatten hierarchial JSON structure?
Hi Forum, I have hierarchical JSON structure coming as input and need to flatten(and transformation) the document as output. Below is the simple example showing input and output structure. I tried doing this with Mapper, Strcuture and JSON Splitter snaps but could not achieve it. Anyone know how to solve this? Input document structure: [ { "entity": { "results": [ { "id": 453721, "custom_fields": [ { "id": 11111, "value": "AAAA" }, { "id": 22222, "value": "BBBB" }, { "id": 33333, "value": "CCCC" } ] } ] } } ] Desired Output Document structure: [ { "entity": { "results": [ { "id": 453721, "11111" : "AAAA", "22222" : "BBBB" } ] } } ]
Issues in reading Nested Json file
Hi, I tried executing the example for Json Splitter provided in their home page, but the results i’m getting does not match with what they provided in their page. Could someone please correct me if i am wrong somewhere. SamplePipeline.slp (6.6 KB) ExpectedResult.xlsx (8.4 KB) Thanks, Muthu