Maximum size of a JSON document

[PSAmmirata]

Is there a limit on the size of a JSON document that SnapLogic supports?

[winosky]

This is awesome @koryknick, this sheds some light. I’m going to assume the JSON Formatter snap is also in that list as that was the one that did it for me. Although I do also have a gate snap but the plex didn’t started falling over until I added the JSON Formatter so it sounds like a combination of memory intensive snaps.
Now that I think about it, it might not even be just memory intensive snaps but enough snaps to cause the heap size to go over it’s limit.
Fortunately I found a better alternative but I’m interested if you developed a method into determining ways to throttle read snaps or is throwing ram at the plex your conclusion?
Thanks

[koryknick]

It is possible that JSON Formatter would contribute, especially if following a Gate that consumes the entire input stream into memory. A JSON Splitter may do the same thing since it reads one document at a time to be able to split… if that document is particularly large, it would need enough memory to load it.

The number of snaps doesn’t necessarily contribute to the in-use memory unless you are passing one very large document and the snaps need to allocate and de-allocate the memory.

My recommended remediation would be pipeline redesign to limit the number and type of memory intensive snaps. As with most things, there are a number of ways to accomplish the same goal. The art of this science is to find the best solution for the given environment.

There are some enhancements to many of the snaps I listed that can help limit the amount of memory being used.

[winosky]

Ok thanks for the insight Kory, appreciate the help.