Redshift Bulk Load on Cloudplex
We have a Redshift cluster that we would like to load data into. Using the Redshift - Insert snap is simple, but prohibitively slow and isn’t the recommended way to load data into a Redshift. We would like to use the Redshift - Bulk Load snap, but are running into a few issues when setting up the Redshift Account in SnapLogic.
- Our understanding is that using IAM Roles for authentication is NOT possible on a Cloudplex. Is this true? If so, this is a huge issue.
- If we can’t use an IAM Role for authentication, the only other option is an AWS Access Key with its secret and token. The main issue with this approach is that the tokens are temporary and only last a few hours at a maximum. How can we use an AWS Access Key with its secret and token without having to refresh the token every 15 minutes? This doesn’t seem useful.
Any help would be great. Thanks!
Expanding on the solution provided and using information from the web related to javascript, I came up with the following statement. There was experimentation involved for sure.
$.mapValues((value,key)=> typeof(value) == ‘object’ && value.hasOwnProperty(“@xsi:nil”)? null :value).mapKeys((value,key)=> key.replace(‘ns2:’,‘’))
mapValues and mapKeys are admittedly a bit of a black box for me still.
Thanks @JensDeveloper for leading me done a good path.
