Showing results for 
Search instead for 
Did you mean: 

Propagate Schema downstream

New Contributor II

We are trying to write files into S3. But we want to write in a specific JSON format so that there is no impact to all the systems that are already consuming an existing structure.
We are trying to use a mapper to map the source to this JSON format.
where and how can I specify the format that I am looking for so that I can map the values in mapper? Is there an alternative approach this can be achieved?



Typically, you would do this by specifying the schema in the snap following the Mapper in the pipeline. For example, in your case, I would imaging that you have a Mapper, then a JSON Formatter, and then a File Writer (to write to S3). The JSON Formatter allows you to specify the schema and this schema will be back propagated to the Mapper.