SQL Server Bulk Load

[asepuri]

Hi All,

Am trying to load large volumes of data from Amazon S3 onto Sql Server Azure 2014. If I use ‘SQL Server Insert’ then it is taking way longer to load a single big table. Is there a way that bulk load option works without ‘BCP Absolute Path’ on cloudplex. Below is the pipeline,

For BCP absolute path, is it just generating a format file(xml or non xml) using bcp utility and uploading it onto snaplogic and referring the path in snap. Is my understanding correct or is there a better way to do this.

Thanks in advance!

-Arun

[ebarner]

Arun,
To use the BCP utility (BULK LOAD) for SQL Server, it does require the SQL Server client tools (or command line utilities) to be installed on the SnapLogic Execute Node. You mention a cloud-plex in your post so please reach out to SnapLogic Support and they can work to provision that in the cloud node(s) for you.

Eric B.

[asepuri]

Thanks. Will try to reach out support.

[asepuri]

@ebarner/@dmiller,

We reached out to support and they said it’s not possible to install sql server client tools on cloudplex.
So, is there a way we could improve ‘SQL Server Insert’ Snap work faster to load records.

Below are the settings/parameters on sql server account,

Thanks,
Arun

[mina]

Hi Arun, I see we do have Azure SQL - Bulk Load which uses SQLServerBulkColpy API
https://doc.snaplogic.com/wiki/display/SD/Azure+SQL+-+Bulk+Load
maybe try it out?