Created by @pkoppishetty
This pipeline is designed for transferring data from a local machine to Google Storage, and then loading data into Google BigQuery using a data source as Google Storage.
Configuration
Make sure- Upload type is:- “Upload existing files from Google Cloud Storage” and file paths:- “file format”:“AVRO”, “file path”:“gs://gcs_cloud_1/data/leads.avro”
Sources: Local machine, Google Storage
Targets: Google BigQuery
Snaps used: File Reader, File Writer, Google BigQuery Bulk Load (Cloud Storage)
Downloads
Pattern to ingest data from Google Storage into Google BigQuery using AVRO file format.slp (5.6 KB)
