Forum Discussion

vaidyarm's avatar
vaidyarm
Contributor
6 years ago

How to receive file through snaplogic API call

hi

we have a requirement to receive a file in snaplogic where we have to provide API URL to an external team. that team will do REST post along with file attached in POST request. we are supposed receive the file and store it. we tried a number of ways but it’s not working.

we sent a text file through Postman on API POST request as an attachment, but that pipeline failed with error msg as below.

“Document input views only accept JSON-encoded data”

let us know if there is a way to consume a file from such REST POST request with files attached to it

8 Replies

  • tlikarish's avatar
    tlikarish
    Employee
    6 years ago

    vaidyarm,

    You can do this with a triggered task, but need to make sure there is only a binary view open, otherwise it will fail with the error:

    Document input views only accept JSON-encoded data

    SnapLogic has two types of input views: document and binary. The document input views are used for structured or semi-structured data like Json, XML, CSV, etc. Document input views are represented by a Snap with an open circle like this.

    For your use case, the request will be multipart-form-data and so the pipeline should have a binary input view. Specifically you should use a Multipart Reader that will be able to extract the binary data from the POST request. The binary input view is an open diamond.

    Here’s an example of a Pipeline that takes the POSTed file and stores it in SLFS using the Multipart Reader and the File Writer. You can try it out by importing this pipeline, creating a triggered task, and then using Postman/curl to send a file to the task endpoint. You should see the file you sent written to a file named ‘attachment.txt’ in the same project as the pipeline.

    post-file-example_2019_10_04.slp (4.3 KB)

    • vaidyarm's avatar
      vaidyarm
      Contributor
      6 years ago

      that really helped !!! i am able to transfer all kind of posted files now wherever I need to send it.
      but there is one thing that i have to pass in API requested along with the attachment, the name of the file in the query field. then i map this pipeline parameter to the filename in file writer
      is there a way where i can simply provide attachment and auth details to the REST POST API and pipeline should extract the name from it in form of some variable which we can map as the file name in file writer snap ?

  • tlikarish's avatar
    tlikarish
    Employee
    6 years ago

    Not really at the moment – although there is now an enhancement ticket to improve the Multipart Read so that it could do things like this. I’ll try and keep you updated about when it goes into development.

    For now, another option would be to write the file to a “temporary” location and then have a separate pipeline that moves the pipeline to the final location. Then the API would require two POST calls, one to upload the file and the other that would take the file path and other things needed to write it to the final location that would “commit” the file.

    • vaidyarm's avatar
      vaidyarm
      Contributor
      6 years ago

      let us know such updates.

      however, as you said we can write file to temp location like sldb, even for that we would require some parameter name to map as the filename.

      Ex: the pipeline you shared is writing file as attachment.txt, not the name of the actual sent attachment, how this can be handled if the name of file need to be the same in post request?
      Also, I tried with making task as ultra and pass parameter, ultra is not capturing the parameter in triggers

  • tlikarish's avatar
    tlikarish
    Employee
    6 years ago

    hey – originally I was thinking that you could just write to that constant file location with maybe a timestamp or uuid to make it unique. If you add an output view to the FileWriter, then it’ll respond to the request with the file locations of where things were written. Alternatively you could also use a regex to read the file names, or other parts of the content-disposition header, that are passed into your task and re-use those.

    Here’s an example pipeline that shows how that second approach would work.
    post-variable-filename-example_2019_10_10.slp (6.1 KB)

  • Rudhiran's avatar
    Rudhiran
    New Contributor
    2 years ago

    HI tlikarish  Above code isn't working for me. I tried to send .xlxs  and .csv file but in mapper snap $['content-disposition'].match('filename=\"(.*)\"')[1] getting error that content-disposition not found

    Appreciate your help on this issue

    Anyone who know the solution can help us - appreciate your help 

    Thanks 

    • tlikarish's avatar
      tlikarish
      Employee
      2 years ago

      Hi Rudhiran -- the double quotes were escaped in my example. Not sure where those came from, but could you try this expression, with backslashes removed, in your mapper and see if it will work for you?

      $['content-disposition'].match('filename="(.*)"')[1]

       

      • Rudhiran's avatar
        Rudhiran
        New Contributor
        2 years ago

        HI tlikarish , 

        Multipart reader gives response only Content type and Content - Not the  content-disposition

        So we're getting error in mapper that 

        Cannot invoke a method on a null value

         

        Resolution:
        Please check expression syntax

         

        Reason:
        Value referenced in the sub-expression '$.get('content-disposition').match('filename="(.*)"')' is null
         
        Appreciate your help on this issue