Move data from a relational database into Teradata
Created by @pavan This pattern pipeline migrates data from a relational database, like MySQL, to Teradata. Configuration Note: Sample database and table names are used within the Snaps to give you an idea on expected values. Replace them with your system information. Sources: MySQL table Targets: Teradata Snaps used: MySQL Select, TPT Load, Teradata Execute Downloads 4.8_TPT FLoad.slp (5.3 KB)
Reference Pipeline : To get schema details for a Teradata Table
As part of ones implementation, one would need to query the metadata/table schema for a database table. The following pipeline is used to get field metadata for a given table in Teradata. You can call it from any pipeline by using the Pipeline Execute Snap and make sure you provide the schema name and table name When calling the above from a master pipeline, it would look like below. In this example, we are passing the table schema from Teradata (for the table) to the input view (second view) of CSV Parser. The CSV Parser Snap has an optional second input view that takes in an input schema (csv of field names). Source: Teradata Target: master pipeline Snaps used: JDBC, Mapper, JSON Generator Download Attached is the reference pipeline to get Teradata Schema Pattern - Teradata - Get columns for Table_2018_06_25.slp (11.1 KB) The master pipeline can be found in the topic: Teradata to Cloud Data warehouse : Teradata to Azure SQL Data WarehouseTeradata to Cloud Data warehouse : Teradata to Azure SQL Data Warehouse
Teradata TPT FastExport Snap enables one to export large amounts of data out of Teradata very efficiently. On the target side if you are moving data into the Azure SQL Data warehouse, use the Azure polybase bulkload. If you were to move data to Redshift, then use the Redshift Bulkload Snap. Below is an example how you can configure the Teradata TPT fast export snap. The prerequisite is that you need to install the Teradata TPT utilities on every node of your Snaplex. Snaps Used: TPT FastExport, File Reader, CSV Formatter, CSV Formatter, Pipeline Execute, Azure Polybase Bulkload Download Teradata to Azure SQL Data Warehouse_2018_06_25.slp (10.0 KB) This pattern also calls the Pattern described in Reference Pipeline : To get schema details for a Teradata Table