Forum Discussion

Roger667's avatar
Roger667
New Contributor III
3 years ago
Solved

Unable to write salesforce data to s3 in parquet format

I Using saleforce read snap to read data and write to s3 in parquet format but getting error due to 'picklist' data type which is not compatible to parquet. How to handle such datat types problem
  • manichandana_ch's avatar
    manichandana_ch
    3 years ago

    Hi Roger667 

    Could you please check your SF read snap and make sure the 'Match datatype' is checked.

    Thanks !

manichandana_ch's avatar
manichandana_ch
New Contributor III
3 years ago

Hi Roger667 ,

you need to enable the second output view of salesforce read snap (schema output) and get the metadata of the columns, convert them to the datatypes compatible with parquet writer and then pass the metadata to the second input view of parquet writer (enable second input view, schema input).
attaching sample pipeline and file used to convert the schema.

 

Thanks,

Mani Chandana Chalasani