07-24-2020 10:47 PM
We are trying to write files into S3. But we want to write in a specific JSON format so that there is no impact to all the systems that are already consuming an existing structure.
We are trying to use a mapper to map the source to this JSON format.
where and how can I specify the format that I am looking for so that I can map the values in mapper? Is there an alternative approach this can be achieved?
08-24-2020 12:55 PM
Typically, you would do this by specifying the schema in the snap following the Mapper in the pipeline. For example, in your case, I would imaging that you have a Mapper, then a JSON Formatter, and then a File Writer (to write to S3). The JSON Formatter allows you to specify the schema and this schema will be back propagated to the Mapper.