Pattern to ingest data from Google Storage into Google BigQuery using AVRO file format

This pipeline is designed for transferring data from a local machine to Google Storage, and then loading data into Google BigQuery using a data source as Google Storage.

Configuration

Make sure- Upload type is:- “Upload existing files from Google Cloud Storage” and file paths:- “file format”:“AVRO”, “file path”:“gs://gcs_cloud_1/data/leads.avro”

Sources: Local machine, Google Storage
Targets: Google BigQuery
Snaps used: File Reader, File Writer, Google BigQuery Bulk Load (Cloud Storage)