cancel
Showing results for 
Search instead for 
Did you mean: 

Google Big Query

TWCMG
New Contributor II

Is it possible to truncate a table in bigquery and then run Bulk Load(Streaming). I’ve tested using BigQuery Execute with truncate statement and then link a bulk load(streaming) snap. I get an error message saying invalid input. Its not recognizing the table schema?

2 REPLIES 2

ddellsperger
Moderator
Moderator

Truncation followed by a load is very discouraged in BigQuery as the truncation is an operation that results in eventual consistency issues in the BigQuery backend (it needs to persist the state of truncation across all backend nodes). Our BigQuery Write and BigQuery Bulk Load (Streaming) will explicitly fail if the table is truncated as it may potentially cause data loss, you can try the BigQuery Bulk Load (Cloud Storage) as it has more appropriate retry mechanisms built in to support truncated table operations.

TWCMG
New Contributor II

Ok makes sense. Are there other ways/alternatives to truncate and load a bigquery table in snaplogic?