Reusable pipeline- Unable to fetch variable value in child pipeline -Snowflake Snap
Usecase- We are looking at creating a reusable pipeline to load data from table1 from source DB to table1 in target DB. Plan is to reuse the same pipeline for n tables. Option 1 - Created master pipeline and child pipeline. Master pipeline gets table name list from source DB. Then uses mapper and foreach snap to execute the child pipeline. Child pipeline loads table from source DB to target DB. Issue- Unable to use the parameter/variable value from master pipeline-foreach snap to child pipeline -snowflake select/execute snap. Master pipeline- Child pipeline- Anyone faced similar error or has recommendation for the approach on reusable pipeline. Thank You.Solved6.7KViews0likes7CommentsAccessing temp table Snaplogic creates in Snowflake?
Is it possible to access the temp table Snaplogic creates in Snowflake? Need to do some deletes and the delete snap takes a lot of time and resources. I would like to delete using a join to the temp table instead of individual transactions if possible. “MERGE_TEMP_TABLE_”4.9KViews0likes6CommentsGetting errors move large data to Snowflake
Hello, We are trying to move large amount of data from MS SQL to Snowflake using external staging, and getting following error. Too many S3 upload parts, maximum number of parts is 10000 Resolution: Increase the buffer size so that the file size divided by the buffer size is smaller than 10000. Reason: The file is too large for the buffer size 10000000 The source table contains 1 billion records. We are using external AWS S3 as staging area. I checked both Snowflake account and Snowflake bulk snap, but cannot find configuration related with buffer size. It will be appreciated if anyone can share the solution for this. Thanks!3.1KViews0likes1CommentSnowflake - Bulk Load giving error
Hi, I am trying to use Snowflake bulk load snap to write data to a Snowflake table from input view. But when I have tried to validate or execute the pipeline I got the following error, Pipeline Execute: Error details: What is this temp file here? Can anyone help me?Snowflake execute (select SQL)- Capture results
Hi, We are using Snowflake execute snap to run a metadata query against Snowflake and using the output of it in mapper snap. Snowflake execute snap- preview data shows expected data from sql. In the mapper snap, input schema is not available for snowflake execute-select sql. Is there a way to capture/use the results from snowflake execute-sql(select) and use it in mapper or subsequent snaps.3KViews0likes2CommentsDoes Snowflake snappack 'Bulk Load' snap support ingesting data on to snow flake managed internal stage?
Snowflake has three kinds of stages before loading data on to snowflake dbs. 1/ Supported cloud storage, 2 / Internal snowflake managed named stage 3/ default table/user stage Bulk Load Snap’s documentation says, it does support PUT command. However I found it hard to make the snap work. It is not clear, if the snap does both operations ( PUT first, and followed by COPY INTO) or the pipeline needs two Buld Load snaps in sequence ( PUT snap followed by a COPY INTO snap) The bulk load snap worked (COPY INTO ) if data is already placed in the internal stage. Pls let us know if there is a writer snap that does snowflake PUT operation on to snowflake internal stage.2.7KViews0likes3CommentsCalling REST APIs with Parameters
I need to be able to call a REST endpoint that takes a UserId as a parameter. I’m curious how I can leverage snaplogic to loop over an undetermined list of userIds from one REST endpoint to call another endpoint passing the user id. Is this even possible in Snaplogic?When Snowflake Fails Over
When an instance of Snowflake fails over to another secondary Snowflake account, is there a way to dynamically have the SnapLogic account find which Snowflake account is the new primary? Is there a way to query which of 2 Snowflake accounts are the primary?Solved2.1KViews0likes1CommentSnowflake Issues: Throws syntax error randomly for drop statements in snowflake execute snap. Also Bulk Load snap fails stating table does not exist
Snowflake Issues: Throws syntax error randomly for drop statements in snowflake execute snap. Also Bulk Load snap fails saying table does not exist with create table checked in. Also table does exist and there is no mistake in table name.1.8KViews0likes1Comment