Hi Tlikarish,
Just one more question, since snaplogic is case sensitive, when I loading the excel file, it will fail due to excel header using mixed upper and lower case, but actually column name in my destination is full upper case. Mapper can definitely fix it, but if I’m trying to make the pipeline dynamic, and reusable, I will need some method can read the excel header row and upper case the whole header dynamically. If there is not possible being done by regular snap, I will try script. Thanks again.
Thanks,
David
Mapper can definitely fix it, but if I’m trying to make the pipeline dynamic, and reusable, I will need some method can read the excel header row and upper case the whole header dynamically.
You can use an expression in the mapper like
$.mapKeys((value, key) => key.toUpperCase())
That should work for any header you want. Here’s more information on the mapKeys function.
EDIT: In the mapper you’ll want to set the Target Path for that expression to $. Here’s the configuration I used in my test.
Hello PSAmmirata
The same issue I am facing right now. my database account batch size is set to 50 and if error is in single record entire batch is getting error out with error message of first record where as other 49 records are valid records without erroneous data (tried and tested by manually removing 1 erroneous record ).
By setting batch size to 1 resolved the issue but now pipeline is taking 20 times more time as earlier ☹️
Kindly suggest me you came across any other better solution that would help me a lot.
Just adding to the chorus here. I am having the same issue. I am working to replace sqlldr, so, I am able to compare the same import between 2 methods. Snaplogic loads fine if the data is perfect. If there is any issue, in my case foreign key exceptions. It errors out a lot of good records with the same error code. The error clearly does not apply to a majority of the records (the parent does exist). The rejects do include the records that are appropriately rejected.
