Pipeline output is un-predictable
Pipeline shows accurate output after Script snap. However when we the output is loaded in lets say CSV file, it loads any random data
Forum Discussion
5 Replies
What are your settings on the CSV Formatter and File Writer?
Those Binary Header settings on the CSV Formatter seem strange. The snap is going to create one binary CSV document containing the ALL of the documents from the script output. So you can’t really set headers containing detail-level values. The binary header is more for stuff like content-type, content-location, etc. Doesn’t explain why it’s creating the output you observed, but might be worth just taking those out and seeing if it makes any difference.
Thanks for the response. So our requirement was not write the data in CSV file, but in Database. But because the Pipeline was loading strange data in DB, we just loaded the data into a file to showcase the issue.
I had a similar problem to what you described. Are you creating the document to put it in? If you are merely adding to the data coming into the snap, you don’t have to do anything. You can merely add the value. But if you are writing a different document out, it can create seemingly random data if you don’t first create a place to put it. I wrote this in python, but I ended up writing a line like:
new = java.util.HashMap()
new was the new document I was writing out. The document coming in was created by SnapLogic, and can simply be updated, so it isn’t so obvious, and isn’t a problem. Anyway, I did that, and the problem went away.
