Forum Discussion

shax's avatar
shax
New Contributor
2 years ago

Ingesting large AWS S3 file(30 GB) into a AWS RDS

Hi, I am trying to start of with S3 select  , however s3 select does seem to have size restrictions upto 128 MB ? 

I can't use S3 browser as all I see is meta data information , I want actual data. How should the pipeline look like.

I tried S3 select using a smaller file of 18 MB but seems to get below error which is not helpful at ll! Any thoughts ?

Could not compile expression: SELECT * FROM s3object (Reason: Encountered extraneous input 's3object' at line 1:14; expecting={<EOF>, '||', '&&', '^', '==', '!=', '>', '<', '>=', '<=', '+', '-', '*', '/', '%', '?', 'instanceof', 'in', 'matches', '[', '(', '.', PropertyRef}; Resolution: Please check expression syntax)

No RepliesBe the first to reply