Has anyone come up with a “simple” process to convert files to Parquet format without the need to use a Hadooplex and without going through HDFS.
I would like to read a text file from S3 and convert it to Parquet back onto S3.
I am not aware of a method to do so at the moment; however, Parquet <-> S3 has been an increasingly requested feature recently and we are actively investigating implementation options, though we don’t have a timeline for release at the moment.
Shayne Hodge
Realize this is an old thread, but just wanted to update. In our release last month (Summer), this functionality was added. https://doc.snaplogic.com/wiki/spaces/SD/pages/1438227/Parquet+Writer
Shayne
