Remove obsolete fields from the mapper snap which uses endpoint
You have a response from an API endpoint that brings a huge number of fields. There are few obsolete fields that you want to remove in the pipeline. What is the best way to handle this in Mapper snap? • Add only the fields to be removed from the Input schema and leave the Target Schema as blank with Pass through. • Add all the required fields from the Input Schema and map to Target Schema • Add all the required fields from the Input Schema and map to Target Schema with Pass through • Add the fields to be removed to the Target Schema and pass null values in the Input Schema with Pass through.
How to avoid double encoding issue for URI in Snaplogic?
While processing a Azure blob storage file in Snaplogic, Snaplogic is not able to read it as its encountering with double encoding issue. I have already tried below method to handle double encoding to fix it and worked well but encountering with other issues due to this. Example URI : “https://myaccount.blob.core.windows.net/sascontainer/sasblob.txt?sv=2015-04-05&st=2015-04-29T22%3A18%3A26Z&se=2017-04-30T02%3A23%3A26Z&sr=b&sp=rw&sip=168.1.5.60-168.1.5.70&spr=https&sig=Z%2FRHIX5Xcg0Mq2rqI3OlWTjEg2tYkboXr1P9ZUXDtkk%3D” Endindex : $data.indexOf(“?”) Startindex : $data.search(“sascontainer /”)+11 Head: $.data.slice(0,$.startIndex) Middle : encodeURIComponent(decodeURIComponent($.data.slice($.startIndex,$.endIndex))) Tail : $.data.slice($.endIndex) Rest GET service URL: Head + Middle + Tail above logic is used in Rest GET to read the service url by decoding and encoding only middle part. However this method is also failing even though it handled double encoding as its decoding everything instead of only decoding hexadecimal values of URL. Is there any other method to decode only those hexadecimal values present in URI and encode whole URI on top of it?Editing JSON for CSVParser to add hundreds of headers
Continuing the discussion from CSV Parser, no headers, hundreds of columns: I edited/added the Column Names to CSVParser snap for snaplogic pipeline and when uploaded it as a new file, all of them were empty however those number of fields were added in the snap. below is how I made chnages to the pipline by opening .slp file in a text editor: "columnList": { "value": [ { "column": { "value": " Number" } }, { "column": { "value": " Suffix" } }]} This only gives empty headers. Please suggest how could I make changes to JSON file also let me know if there are better ways to write Column Names without having to add them manually? Thanks!
XML parser is not able handle UTF-8 Character set
I am trying to give input to XML parser after transcoding into UTF-8 format but the special character is getting converted to the garbage value Input : Before XML parser <?xml version="1.0" encoding="UTF-8"?> <UTF_encoded>Š</UTF_encoded> <UTF_encoded1>Montréal,</UTF_encoded1> <street_address></street_address> Output AfterXMLpeaser “UTF_encoded”: "Å " “UTF_encoded1”: “Montréal,” “zipcode”: “11767” “street_address”: "神奈å·çœŒå·å´Žå¸‚ä¸åŽŸåŒºæœ¨æœˆ4-34-1, ｳﾞｨﾙﾇｰﾌﾞ405, " “full_home_address”: “神奈å·çœŒå·å´Žå¸‚ä¸åŽŸåŒºæœ¨æœˆ4-34-1” “national_identifier_type”: “神奈å·çœŒå·å´Žå¸‚ä¸åŽŸåŒºæœ¨æœˆ4-34-1” Can someone please help
Parser issue with size
I am trying to parse 600MB files using json parser, file reader reads the file but json parser runs into following error - Failure: Cannot parse JSON data, Reason: Exception while reading document from the stream, SLDB does not support data larger than 100 MB, Resolution: Please check for correct input JSON data. What is the best practice here to parse large json file ? what is the role of SLDB here ? What is being stored in SLDB here ? even in the architecture document, there is no mention of SLDB here. Can some one clarify the details here ?
Aggregate Snap example
This is an example to illustrate the Aggregate Snap with Functions like Sum, Count, Min, Max and Average. The 1st Snap generates the required input data like Name, Age, Dept, etc. The next Snap “Aggregate” is configured with functions like Sum of Age for Total Age Count of Name (Unique column) for Total employee Min of Age for Minimum Age Max of Age for Maxiumum Age Avg of Age for Average Age Below is the quick screen shot of the configuration of the functions of this Snap. Attached is the .slp for the pipeline reference of this example.Aggregate_2017_03_11.slp (6.0 KB)An XML Generator & XML Formatter Snaps example
Here’s an example illustrating the use case of the snaps XML Generator, XML formatter and File writer. The 1st snap “XML Generator” of the pipeline generates a document format data with the given xml input configured in the snap on “Edit XML *” in snap settings. The 2nd snap “XML Formatter” accepts the document format and encapsulates the data into XML formates and provides the the binary output which can be an input to the File writer snap which accepts the binary input. Attached is the .slp for pipeline reference.XMLGenerator_Formatter_2017_03_1.slp (4.1 KB)