Parser issue with size

Question

I am trying to parse 600MB files using json parser, file reader reads the file but json parser runs into following error -
Failure: Cannot parse JSON data, Reason: Exception while reading document from the stream, SLDB does not support data larger than 100 MB, Resolution: Please check for correct input JSON data.
What is the best practice here to parse large json file ?  what is the role of SLDB here ? What is being stored in SLDB here ? even in the architecture document, there is no mention of SLDB here.
Can some one clarify the details here ?

psadasivam · Answer

@ryagnik, From where you are reading the file? did you upload file to your SnapLogic project ?

ryagnik · Answer

I am reading the file from file server and use file reader snap to read the file.

aleung · Answer

can you share the “File name” value you have in your File Reader snap?

ryagnik · Answer

the name of the file is out.json.

ryagnik · Answer

please note that this only happens in preview mode, I was able to execute the pipeline with large file.

Forum Discussion

Parser issue with size

8 Replies

Recent Discussions

SFTP: Worked ok then settings changed not not working

SQS Consumer snap is ready all message at pipeline startup

Salesforce Update REST API Throughput

Using Windows Authentication with SQL Server Accounts

Preview not available for Flow snaps