โ05-12-2021 11:05 AM
Iโm trying to use an API to post data to an application.
There is no account. Instead I have a specific account.
The curl API looks like this:
curl https://communicate.modolabs.net/api/v1/directories/revisions --header โAuthorization: Token myTokenโ --form โname=Modo Testโ --form โupload=@directory.xmlโ where โmyTokenโ would be the token I was given.
How do I convert this to the REST Snap?
โ08-11-2021 11:29 AM
@mike.korcynski
Not sure on this, but youโre probably right. I will investigate! I likely wonโt be able to look into this today, but tomorrow (Thu, 8/12) for sure.
โ08-13-2021 03:30 PM
@mike.korcynski @Sowmya_Rayavarapu @robert_parks
Here is a solution to POST-ing a file with multiple form fields.
Our REST POST snap is a swiss-army knife, and I am less familiar with all of its options (multipart related, for example). I need to better understand the โentityโ plumbing too. That said, letโs evaluate something that might work โ maybe not an awesome solution, but hopefully something that does work.
We want to POST more than one thing, so letโs ignore the โsingle file uploadโ fields (4 of them). Weโll leave all the entity fields empty too.
The conceptual model is that we need to create a file for every field we want to submit. If we want to upload a file and also specify two form fields, weโll create 3 files! Files can be referenced in SLDB, or the filesystem which may be less convenient for you depending on your Snaplex filesystem access.
Hereโs the top half of the REST POST snap showing the fields I left empty.
Iโve attached two simple pipelines each with a single REST POST snap.
For the first pipeline we want to upload a file (directory.xml) and also specify two form fields. Hereโs the curl version:
curl -X POST http://127.0.0.1:8001/test \
--form "upload=@directory.xml" \
--form "field1=field1-value" \
--form "field2=field2-value"
We can define the form fields as files using the form field name as the โUpload-file keyโ. We then specify a file which contains the form fieldโs value. field1.txt stores the string โfield1-valueโ. We also specify the content type as text/plain.
Hereโs how the POST is received in my test server.
> POST /test
> REQUEST HEADERS:
Accept=[*/*]
Connection=[Keep-Alive]
Host=[127.0.0.1:8001]
Transfer-encoding=[chunked]
Content-type=[multipart/form-data; boundary=PusHivUIyaSW4MgQOpnzZn0wvMu-KHt2]
> REQUEST BODY:
--PusHivUIyaSW4MgQOpnzZn0wvMu-KHt2
Content-Disposition: form-data; name="upload"; filename="directory.xml"
Content-Type: application/xml; charset=UTF-8
Content-Transfer-Encoding: binary
<directory name="foo">
<file>file1.txt</file>
<file>file2.json</file>
</directory>
--PusHivUIyaSW4MgQOpnzZn0wvMu-KHt2
Content-Disposition: form-data; name="field1"; filename="x"
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: binary
field1-value
--PusHivUIyaSW4MgQOpnzZn0wvMu-KHt2
Content-Disposition: form-data; name="field2"; filename="x"
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: binary
field2-value
--PusHivUIyaSW4MgQOpnzZn0wvMu-KHt2--
Ok, for the second pipeline we get a bit more ambitious and double it, that is, upload 2 files, and specify 4 form fields. The second file in this case is an image (a SnapLogic logo, of course!). We also specify some of the form field files on the filesystem (ie, file:///tmp). The 4th form field is defined as json. Thatโs a nice part of multipart upload, each part defines its content type.
Iโve attached all the assets and pipelines for above, including the request POST output for the 2nd pipeline POST against my test server.
There may be some more clever, more elegant plumbing that others can share โ please do! I was just focusing on how to get something to work for you (and me, and others).
Do let me know if this helps, or if doesnโt, where you are having problems. Many of our snaps wrap REST APIโs so you donโt have to think about these details.
rest-post-1-file-2-form-fields_2021_08_13.slp (3.7 KB)
rest-post-2-files-4-form-fields_2021_08_13.slp (4.6 KB)
rest-post-example-assets.zip (7.7 KB)
โ08-16-2021 08:01 AM
Hi @mbowen,
Thank you for the response. You have specified the files in rest post snap. It is easy to specify the files in rest post snap, if we know the number of files and their content-types in advance.
But in my case, I donโt have a clue about number of files I receive and their content types. The file number vary from 1 to 10 and their content type also varies. Could you please help me with the request?
โ08-16-2021 04:44 PM
Tell me a little about the service(s) youโre invoking? Are they custom in-house services that you control, or external? Our REST snap pack is not a premium pack, so while it provides a fairly broad range of functionality it may be a bit rough around the edges for some use cases. What Iโm getting at is that itโs not as light weight as curl, where I can see dynamically creating POST requests without too much effort.
I donโt have much experienced with form-related data, so Iโm not sure if that could help here.
Representing a form field key/value as a file seems pretty heavy to me, but weโre piggybacking on the multi file upload functionality. Most of the fields in an upload file row are expression enabled, so could be supplied from either input data or pipeline parameters.
If you controlled the backend, I suppose you could pack all non file form field data into one file and then decode that in the backend. Again, not pretty, and I wonโt even suggest this. Itโs not obvious how to inject an upload file table into this snap.
That said, Iโm often amazed at other peopleโs pipeline solutions. I wonder if any folks are using scripting for this โ probably not for file upload, but maybe. Our Script snap supports a variety of languages (Python, JavaScript, โฆ) with limitations though.