Pattern to ingest data from Google Storage into Google BigQuery using AVRO file format
Created by @pkoppishetty This pipeline is designed for transferring data from a local machine to Google Storage, and then loading data into Google BigQuery using a data source as Google Storage. Configuration Make sure- Upload type is:- “Upload existing files from Google Cloud Storage” and file paths:- “file format”:“AVRO”, “file path”:“gs://gcs_cloud_1/data/leads.avro” Sources: Local machine, Google Storage Targets: Google BigQuery Snaps used: File Reader, File Writer, Google BigQuery Bulk Load (Cloud Storage) Downloads Pattern to ingest data from Google Storage into Google BigQuery using AVRO file format.slp (5.6 KB)
Dates to BigQuery problem
I’m trying to export some dates into BigQuery, but even though the preview of the data that is being sent to BigQuery is correct, when I check my table in BigQuery it always shows as null. Any idea on how to solve this? My data is parsed this way: Date.parse($ACCT_STATUS_DATE.Date,“yyyy-MM-dd”)
