Create Avro Dynamic Schema
Hi Team,
I would like some recommendations/directions regarding possible solution to below requirement.
I have created a generic pipeline to read 800 different Mainframe VSAM binary files with di...
Forum Discussion
If you generate a separate Avro schema for each data file, you could save the schema to a file on sldb, then invoke a child pipeline using Pipe Exec, setting a pipeline parameter to specify the Avro schema file’s path. In the Avro Formatter in the child pipeline, the schema setting would be an expression you’d set to the value of the pipeline parameter.
Thank You @ptaylor … I am looking directions on ways to generate Avro schema dynamically using data file or copybook structure. Once the schema is generated and created in sldb I can sue above method to pass that to child pipeline so that Avro formatter can use it while creating Avro target file in S3.
