Forum Discussion

venkat's avatar
venkat
New Contributor
6 years ago

Snowflake Bulk Load

Hi, I am trying to use Snowflake bulk load snap to read data from external S3 location and load into the snowflake table. here the challenge is that. Under the Snowflake account, I have mentioned...
venkat's avatar
venkat
New Contributor
6 years ago

Yes, pipeline ran successfully but data is not loaded. I am able to load data into table using s3 read and then snowflake insert snap which i am trying to avoid .

As per the Snowflake History:-

smanoharan's avatar
smanoharan
Employee
6 years ago

After reading the data from s3 read, Instead of Snowflake Insert Snap, you can use the Snowflake Bulk Load Snap and select the Data Source as Input view. I would like to know if the Bulk Load snap is able to process that data.

Seems like the History doesn’t have any bulk api executions (“copy into”). can you attach your pipelines after removing the sensitive data.

  • smanoharan's avatar
    smanoharan
    Employee
    6 years ago

    I took a look at the pipeline. The root cause of the issue seems to be the case sensitivity of the column headers. Can you sync up with the support contact, Eric on this.

    • venkat's avatar
      venkat
      New Contributor
      6 years ago

      thank you
      I am able to fix it. What if I want to use External staging file (S3) instead of S3 file reader

      • smanoharan's avatar
        smanoharan
        Employee
        6 years ago

        As long as you make sure that file contains the column headers in the same case as present in the Snowflake Database table, it should not be a problem in most cases.

        In your case, you have said that the staged Data and Snowflake are on different AWS accounts. We have to figure out if there are any gaps there.
        In the Snowflake account, did you give the Amazon S3 credentials of the staged data?