Forum Discussion

asepuri's avatar
asepuri
New Contributor II
9 years ago

SQL Server Bulk Load

Hi All,

Am trying to load large volumes of data from Amazon S3 onto Sql Server Azure 2014. If I use ‘SQL Server Insert’ then it is taking way longer to load a single big table. Is there a way that bulk load option works without ‘BCP Absolute Path’ on cloudplex. Below is the pipeline,

For BCP absolute path, is it just generating a format file(xml or non xml) using bcp utility and uploading it onto snaplogic and referring the path in snap. Is my understanding correct or is there a better way to do this.

Thanks in advance!

-Arun

5 Replies

  • asepuri's avatar
    asepuri
    New Contributor II
    9 years ago

    @ebarner/@dmiller,

    We reached out to support and they said it’s not possible to install sql server client tools on cloudplex.
    So, is there a way we could improve ‘SQL Server Insert’ Snap work faster to load records.

    Below are the settings/parameters on sql server account,

    Thanks,
    Arun

  • ebarner's avatar
    ebarner
    Former Employee
    9 years ago

    Arun,
    To use the BCP utility (BULK LOAD) for SQL Server, it does require the SQL Server client tools (or command line utilities) to be installed on the SnapLogic Execute Node. You mention a cloud-plex in your post so please reach out to SnapLogic Support and they can work to provision that in the cloud node(s) for you.

    Eric B.

    • asepuri's avatar
      asepuri
      New Contributor II
      9 years ago

      Thanks. Will try to reach out support.

  • ebarner's avatar
    ebarner
    Former Employee
    9 years ago

    Arun,
    I have confirmed with with our PM organization that customization (installing client utilities etc.) to CloudPlex enviroment is not possible. Previously I thought this was an option and I apologize for leading you down the wrong path.

    You may want to try turning off AutoCommit to see if that improves performance at all.