Dynamically change the Delimiter in CSV parser
I’m designing ah pipeline that will handle multiple delimiters in a single CSV parser. I’m using a expression library that has the delimiter details of the respective files as below. [ { “src_file”: “aab”, “Delimit”: “PIPE”, “tgt_table”: “T_STG_AAB” } }, { “src_file”: “abc”, “Delimit”: “TAB”, “tgt_table”: “T_STG_ABC” } }, { “src_file”: “efg”, “Delimit”: “COMMA”, “tgt_table”: “T_STG_EFG” } } ] On the mapper snap I can write expression to refer this expression library lib.sample.find(x => x.src_file==“incoming_filename”).get(‘Delimit’). But when I try to use the same in CSV parser it doesnt accept. can we really do this.
Exporting text file with CSV formatting, but with different numbers of fields
I’m working on a pipeline that needs to create and send a text file to a third party That text file contains rows formatted in the same was as a csv file The file contains a header row, then the data rows, then a footer row The header and footer rows only have 6 columns, and I cannot send it with more or less columns, because their system will reject it the data rows have 33 columns, and I cannot send it with more or less columns, because their system will reject it Here is my pipeline: The first section has 3 SQL Server Execute snaps that get the 3 types of rows, then I union them all together. The select contains 2 fields that cannot be in the resulting text file, and I only need them for the sort snap, to get the rows in the correct order The mapper after to Sort snap is to remove the 2 sort columns The problem I get here is, if I leave null safe access unchecked, then it freaks out because Col6 to Col33 does not exist in 2 of the rows, and if I check null safe access, it creates 6 to 33 in those 2 rows and adds to many fields to those 2 rows in the text file Is there any way to: A) Remove the 2 fields without using a mapper B) Remove the resulting null valued fields after the mapper OR C) Tell the csv formatter to not create the field if it has a null value Thanks
Repeat Target Path/ Header Name in a csv file
Hi, My CSV has headers which repeat themselves - BLANK2 , BLANK2, BLANK2 , NAME, BLANK2 Mapper gives me error saying that I can’t have same Target Path names. I tried adding header fields to CSV formatter and Binary Header, that didn’t work. I tried adding .string() after the field name- didn’t work I am thinking of a very convoluted way of doing this, but wondering if there is an easy fix that I am unaware of.
CSV Parser Troubling Error (Cannot complete CSV data parsing)
Hi, I am trying to parse a CSV that is auto-generated by a legacy system. The CSV has no headers. The problem is, the system conditionally adds columns to individual records. So, the CSV sometimes has records with more columns than others. This causes Snaplogic’s CSV Parser to fail (it doesn’t parse the whole file). The parser will only parse up to the row number where the record with the additional columns is. Below is the error message: This sounds far-fetched but is there a way to automate a solution to this problem? Perhaps by being able to conditionally control the columns before the document hits the CSV Parser?
Append data to csv file with out repeating the header row
hi there, i am trying to append all the records for the day to that days CSV file which has date in the file name. all works fine, if it doesnt fnds the file for the day it creates the file. if it finds the files then it appends the data to the file. But one thing is keeps adding the column names as well to the file. How can i have the only data be appended and not the column headers? Thanks Manohar
CSV Parsing issue
As per documentation CSV parser settings Escape character says : Leave this property empty if no escape character is used in the input CSV data. However when escape character '' is present in the data it fails. Only if I add \ in escape character it runs successfully however it removes \ within the data. Did anyone faced this and how to get read of this issue?CSV Formatting issue
Hi I am getting json data in attached format from a JOIN Snap when viewed in table, I want to make one CSV out of it but it is taking only headers of the first json object screenshot of which is attached below. JSON form output Table form output Need help in how can I use column headers as the key and its value in a single CSV. Sample data preview of SNAP CSV FORMATTER What is the change that I have to make so that I can get all headers as previewed in the second screenshot? Any help is appreciated.
I'm completely stuck trying to write JSON to CSV - it's been 4 days. Can anyone help?
I’m brand new to Snaplogic and this is probably really easy - sorry in advance! But I’ve been stuck on this since Thursday, and it’s now Tuesday… please help me. As a whole, I have this: In more detail: I have a JSON splitter, which has an output of the JSON I want to write to a CSV. And, if I preview the output in table view, it looks exactly like the CSV I want (I had an image here, but new users can only put one - sorry!). Previewing in JSON shows that it’s, well, JSON. After that, I have a CSV formatter, but I don’t have anything configured in it. Should I? The output view for this is more or less empty, so this clearly isn’t correct. I’ve tried JSON formatters and a million other things here. I just don’t know. Then I have my file writer (and as a bonus, I want only some of the fields from the JSON to be written to the CSV file). I can do it in Python: # Call the API r = requests.get(url_to_call, headers = headers) json_to_write = r.json() # Open a file for writing outfile = open(file_name,'w') writer=csv.writer(outfile) # Write CSV Header Row writer.writerow(["Id", "FirstName", "LastName"]) # Loop through each line of the JSON; print key values needed for x in json_to_write: writer.writerow([x["Id"], x["FirstName"], x["LastName"]]) I found this post on turning JSON to CSV by @dmiller and this post on writing a Twitter Query to a file, but as far as I can tell, those are just “use these snaps” without showing how the snaps are configured, and I’ve tried those configurations of snaps. I’m so lost. Any help appreciated.