Output from script snap (Python) is writing incorrect

[aditya_sharma]

Hi,

I am using script snap (python) to split comma separated values within one of my source column and making a dynamic update statement from it. However, when I am doing a preview of script output, all the values looks correct but if I look the preview in mapper output which takes input from script snap shows different output i.e. sometimes all the update statements are same and sometime two are same and one is different.

Am I doing something wrong?testchild_2017_09_22.slp (9.2 KB)
iods_record_1.zip (1.1 KB)

[dmiller]

Can you clarify in what way they are different?

[akidave]

A document should not be modified once it has been written to the output view. Making a copy of the document avoids issues. Change the loop to create a new wrapper Map

         for item in sourceTables:
             wrapper = java.util.HashMap(in_doc)
             wrapper['query'] = "update iods_staging.{} set record_inserted = true;".format(item)
             self.output.write(wrapper)

[aditya_sharma]

Hi Akidave,

Thanks you so much for your solution. However, I would really appreciate if you could explain me more why we should not modify the document once it has been written to the output view.

Thanks again.

Aditya

[akidave]

aditya.sharma:

why we should not modify the document once it has been written to the output view.

If a snap writes a document to output view and then modifies the document, the downstream snap might pick up the updated document value, not the document originally written. For performance reasons, a deep copy is not done automatically when the document is written to the output view.

Creating a Map in the script works in this case since the Script is changing only the top level keys. If the Script was changing the contents of the nested map, then a deep copy would be required in the Script