Saving csv in UTF-8 by default
Hi Gooday,
I have a requirement to save the csv file in UTF-8 by default . However in csv formatter we have UTF-8 encoding option , on writing the file in “File Writer” using externsion.csv it is ...
Forum Discussion
I would think this REST service should be doing some sort of quoting for the field with the XML in it so this wouldn’t be a problem. Is that not happening? Can you give an example of the full row of data?
The ELT DML snaps (namely the LOAD, INSERT-SELECT, MERGE INTO, SCD2, EXECUTE) were intentionally designed to not execute in Validation mode (Design mode) for the following reasons:
- These snaps can inadvertently corrupt or make the data invalid on production target tables if somebody runs validation accidentally on those pipelines. We want to prevent such mishaps from occurring.
- Validation mode (Design mode) can run fast when DML snaps are not writing into the target tables, otherwise it will not only increase the performance overheads, but also increase CDW costs.
In order to understand your problem better, can you provide some example use cases? This will help us in addressing the problem.
I understand your 2nd point and that is something a customer’s development team should be considering anyway to ensure the usage costs are managed.
As for your first point other than the obvious push down behavior why would these snaps behave different than to other DML snaps in other snap packs? Isn’t “Design Mode” just doing a limited run of the pipeline snaps? The only way to validate your pipeline is to run it and if there are any exceptions you need to be an admin in the environment to see what Snaplogic has sent to the DB. We have already discussed this with your product team and they told us this was a large level of effort and needed more customers to request this behavior to put it into roadmap, they never mentioned the corruption issue. We would never validate or go into design mode in a production environment any.
