Forum Discussion

MEsponda's avatar
MEsponda
New Contributor
3 years ago

How to write a CSV using HDFS Writer

Hello!
I have a issue,

When I try to write a csv using the HDFS Writer… the snap aborted
So… the question is if someone know what is the limit size of the file that I can write using the HDFS Writer?

Because the file that I want to write is about 10 GB.

Blog
expression_language
pipeline
sample_pipeline
transform

1 Reply

  • Swati_Sahoo's avatar
    Swati_Sahoo
    Employee
    3 years ago

    Hi @MEsponda

    May I know ,you are trying for which protocol in HDFS writer snap? if it’s ADLS gen 2 (ABFS), snap is writing a 10 GB CSV file to abfs without any issue.