REST POST A File?
Seems like a silly question, but I have a pipeline with a REST POST snap like below which posts a file pulled directly from the SLDB filesystem and posts it to a URL, specifying an Upload File and Up...
Forum Discussion
Hello,
Due to Java memory segment limitations, files being posted to an endpoint that require a multipart post must be streamed from a file.
Please note that a file upload with the REST post snap equates to a multipart post, which is not the same as trying to post a string of binary in the body which would be just a standard post, not multipart. Most API’s which are expecting files do not expect the latter and this would not work for them.
REST Post requires the file to be localish(something it can access without credentials) so that it can stream the data from the file to the endpoint. It is unsafe and unperformant to essentially build a big string of binary and try to post it somewhere, multipart or not.
If your string is too large it can (1) crash your snaplex node requiring a reboot (2) not work because part of its missing and lead to a corrupted file (3) Even if 1 or 2 isn’t a problem lead you to file-encoding blackhole and corrupted files because you’re translating in (and possible out) of UTF-8 From whatever source encoding that’s coming from.
Interesting…
So, if my pipeline already has a file, say something posted in via triggered event, or maybe created in the pipeline itself via a formatter type snap - then I would need to first write that binary file to local FILE/SLDB storage, and then reference it in the REST POST snap if I wanted to post that file? If so, the security implications are unfortunate.
If that’s required, maybe the snap itself should handle writing the binary data to a temporary file and then streaming from it within it’s own “black box” logic … Maybe there could be a new BINARY REST POST snap…
