Forum Discussion

makeshwar's avatar
makeshwar
New Contributor II
5 years ago
Solved

Redshift bulk load

Hi team, Looking for a suggestion/enhancement to achieve the below senario. when i read a csv file (it might have \n\r (windows) & \n(unix) ). Who to fix this … Today I read the file and when I d...
etl
redshift
s3
transform
  • makeshwar's avatar
    makeshwar
    5 years ago

    so … this gave me the solution

    $content.contains(‘\r\n’)?
    ($content.replaceAll(‘\r\n’,‘\n’) && $content.replaceAll(‘\n’,‘’) ) : $content.replaceAll(‘\n’,‘\n’)

makeshwar's avatar
makeshwar
New Contributor II
5 years ago

Well, thanks for coming back @mbowen … my target is redshift … I will be getting the file from the win / Unix system and blow will be its behaviour
Win

Unix

I am trying to read the csv file irrespective of the system

i am using the blow code for replacing the \r\n – ($content.replaceAll(‘\r\n’,‘\n’) && $content.replaceAll(‘\n’,‘’) )

Problem - when I pass the Unix file it replaces the \n to ‘’ and makes the csv invalid.

how to accommodate them both

makeshwar's avatar
makeshwar
New Contributor II
5 years ago

so … this gave me the solution

$content.contains(‘\r\n’)?
($content.replaceAll(‘\r\n’,‘\n’) && $content.replaceAll(‘\n’,‘’) ) : $content.replaceAll(‘\n’,‘\n’)