cancel
Showing results for 
Search instead for 
Did you mean: 

Propagate Schema downstream

rajendraj
New Contributor II

We are trying to write files into S3. But we want to write in a specific JSON format so that there is no impact to all the systems that are already consuming an existing structure.
We are trying to use a mapper to map the source to this JSON format.
where and how can I specify the format that I am looking for so that I can map the values in mapper? Is there an alternative approach this can be achieved?

1 REPLY 1

rpatrick00
Contributor

Typically, you would do this by specifying the schema in the snap following the Mapper in the pipeline. For example, in your case, I would imaging that you have a Mapper, then a JSON Formatter, and then a File Writer (to write to S3). The JSON Formatter allows you to specify the schema and this schema will be back propagated to the Mapper.