Forum Discussion

christwr's avatar
christwr
Contributor III
8 years ago

REST POST A File?

Seems like a silly question, but I have a pipeline with a REST POST snap like below which posts a file pulled directly from the SLDB filesystem and posts it to a URL, specifying an Upload File and Upload File Key. This seems to work fine…

But my file won’t really be coming from the SLDB filesystem, so I want to pretend my file is coming in via prior snaps in the pipeline and reference that in the REST POST using the HTTP Entity. So instead I’m loading the file separately and then running it through a BASE64ENCODE, mapping that to the same Upload File Key and then trying to REST POST that, but doesn’t work.

Any ideas?

14 Replies

  • andrew_holbrook's avatar
    andrew_holbrook
    New Contributor III
    8 years ago

    Hi Chris,

    I’ll let the SnapLogic folks reply officially, but a while back I made a REST Post “Binary Form Input” custom snap that’d probably fit your use case. Happy to share and problem solve with you to get it to work for you.

    Regards,
    Andrew

    • christwr's avatar
      christwr
      Contributor III
      8 years ago

      Hi Andrew,

      Yes please. If you have an example, I would love to check it out.

      Thanks!
      Chris

    • Sowmya_Rayavara's avatar
      Sowmya_Rayavara
      New Contributor III
      4 years ago

      Hi Andrew,

      Do you know how to send multiple files to rest post snap? The file numbers are dynamic so I cannot add them in rest post snap, we can expect a minimum of 1 file to maximum of 10 files. All the files needs to be sent in 1 rest call and file content types also vary(CSV,excel,jpeg…)

  • nganapathiraju's avatar
    nganapathiraju
    Former Employee
    8 years ago

    In the document conversion snap you dont need to encode. Just specify document type as conversion.

    • christwr's avatar
      christwr
      Contributor III
      8 years ago

      Hi Naveen,

      Thanks for the suggestion. I’ve actually messed around with all the options in the Binary-to-Document snap, and it doesn’t work with any of them. I cannot specify “DOCUMENT” as the conversion type because it is a BINARY (.wav) file. I can choose NONE or ENCODE, but neither of those work with the downstream post.

      On the REST POST snap, I can specify an Upload File directly (from FILE or SLDB only) and it will correctly construct and post a multi-part mime message using the file. I’m trying to achieve similar capability with a binary file already being processed in the pipeline, not pulled from FILE/SLDB at that point.

      • dwhite's avatar
        dwhite
        Employee
        8 years ago

        Hello,

        Due to Java memory segment limitations, files being posted to an endpoint that require a multipart post must be streamed from a file.

        Please note that a file upload with the REST post snap equates to a multipart post, which is not the same as trying to post a string of binary in the body which would be just a standard post, not multipart. Most API’s which are expecting files do not expect the latter and this would not work for them.

        REST Post requires the file to be localish(something it can access without credentials) so that it can stream the data from the file to the endpoint. It is unsafe and unperformant to essentially build a big string of binary and try to post it somewhere, multipart or not.

        If your string is too large it can (1) crash your snaplex node requiring a reboot (2) not work because part of its missing and lead to a corrupted file (3) Even if 1 or 2 isn’t a problem lead you to file-encoding blackhole and corrupted files because you’re translating in (and possible out) of UTF-8 From whatever source encoding that’s coming from.

  • SandeepVemula's avatar
    SandeepVemula
    Contributor
    8 years ago

    @dwhite, I also need the pipline/script

  • mbostrom_dtiq's avatar
    mbostrom_dtiq
    New Contributor II
    4 years ago

    @christwr did you ever find a solution here? i have a file on AWS S3 and need to include that into an API with an attachment parameter. but the API keeps telling me “{“message”:“The given data was invalid.”,“errors”:{“attachment”:[“The attachment must be a file.”]}}”. not sure how to post a file the correct way from snap after it reads it.