Accessing temp table Snaplogic creates in Snowflake?
Is it possible to access the temp table Snaplogic creates in Snowflake? Need to do some deletes and the delete snap takes a lot of time and resources. I would like to delete using a join to the temp table instead of individual transactions if possible. “MERGE_TEMP_TABLE_”
Move data from AWS S3 to Snowflake Data Warehouse
Created by @pkoppishetty The pipeline pattern moves data from AWS S3 to Snowflake Data Warehouse. The pattern contains a Directory Browser (to retrieve the file list from S3), Snowflake Execute, and Snowflake Bulkload Snaps (to load the corresponding files into Snowflake). Configuration File Reader Preview Snowflake Bulkload Output Table Data from Snowflake Select Snap Sources: AWS S3 Targets: Snowflake Snaps used: Snowflake Execute, Directory Broswer, File Reader, Snowflake Bulk Load, Snowflake Select Downloads Integration pattern for moving data from AWS S3 to Snowflake datawarehouse.slp (14.2 KB)Schema Bulk Load from SQL Server to Snowflake
Created by @ebarner This pipeline loads the schema from the specified SQL Server table into the Snowflake table. Configuration Sources: SQL Server table Targets: Snowflake table Snaps used: SQL Server - Select, Snowflake - Bulk Load Downloads Schema Bulk Load.slp (4.5 KB)Getting errors move large data to Snowflake
Hello, We are trying to move large amount of data from MS SQL to Snowflake using external staging, and getting following error. Too many S3 upload parts, maximum number of parts is 10000 Resolution: Increase the buffer size so that the file size divided by the buffer size is smaller than 10000. Reason: The file is too large for the buffer size 10000000 The source table contains 1 billion records. We are using external AWS S3 as staging area. I checked both Snowflake account and Snowflake bulk snap, but cannot find configuration related with buffer size. It will be appreciated if anyone can share the solution for this. Thanks!Snowflake - Bulk Load giving error
Hi, I am trying to use Snowflake bulk load snap to write data to a Snowflake table from input view. But when I have tried to validate or execute the pipeline I got the following error, Pipeline Execute: Error details: What is this temp file here? Can anyone help me?Snowflake execute (select SQL)- Capture results
Hi, We are using Snowflake execute snap to run a metadata query against Snowflake and using the output of it in mapper snap. Snowflake execute snap- preview data shows expected data from sql. In the mapper snap, input schema is not available for snowflake execute-select sql. Is there a way to capture/use the results from snowflake execute-sql(select) and use it in mapper or subsequent snaps.
Does Snowflake snappack 'Bulk Load' snap support ingesting data on to snow flake managed internal stage?
Snowflake has three kinds of stages before loading data on to snowflake dbs. 1/ Supported cloud storage, 2 / Internal snowflake managed named stage 3/ default table/user stage Bulk Load Snap’s documentation says, it does support PUT command. However I found it hard to make the snap work. It is not clear, if the snap does both operations ( PUT first, and followed by COPY INTO) or the pipeline needs two Buld Load snaps in sequence ( PUT snap followed by a COPY INTO snap) The bulk load snap worked (COPY INTO ) if data is already placed in the internal stage. Pls let us know if there is a writer snap that does snowflake PUT operation on to snowflake internal stage.MySQL to Snowflake
Created by @pavan Moving relational data (order) in MySQL to Snowflake data warehouse, and later can run analytics on how many orders were fulfilled. Configuration Sources: MySQL Order Targets: Snowflake Snaps used: MySQL, Mapper, Snowflake Bulk Load Downloads MySQL to Snowflake.slp (5.8 KB)
