MattB
New Contributor II
Joined 3 years ago
User Profile
User Widgets
Contributions
Grouping many individual REST requests into a single REST $batch POST
Does anyone have any experience of batching many individual REST requests within a single odata $batch REST POST avoiding the set up costs associated with making a large number of individual requests. I can do so within Snaplogic by grouping and then transforming multiple REST requests into a large string containing the batched requests to be sent within a single REST POST, but wanted to check I haven’t missed something obvious with either the REST POST or HTTP Client Snaps that can be used to do so automatically. Example REST POST, containing a batch of two GET requests - my use case is several thousand POST, PATCH or DELETEs. POST https://fabrikam.sharepoint.com/_api/$batch HTTP/1.1 Authorization: Bearer <access token omitted> Content-Type: multipart/mixed; boundary=batch_e3b6819b-13c3-43bb-85b2-24b14122fed1 Host: fabrikam.sharepoint.com Content-Length: 527 Expect: 100-continue --batch_e3b6819b-13c3-43bb-85b2-24b14122fed1 Content-Type: application/http Content-Transfer-Encoding: binary GET https://fabrikam.sharepoint.com/_api/Web/lists/getbytitle('Composed%20Looks')/items?$select=Title HTTP/1.1 --batch_e3b6819b-13c3-43bb-85b2-24b14122fed1 Content-Type: application/http Content-Transfer-Encoding: binary GET https://fabrikam.sharepoint.com/_api/Web/lists/getbytitle('User%20Information%20List')/items?$select=Title HTTP/1.1 --batch_e3b6819b-13c3-43bb-85b2-24b14122fed1-- Any thoughts or similar problems solved within the community? Thanks, Matt.1.3KViews0likes0CommentsRe: Access file in Python script
The Jersey payload should execute without any other components and the python script will unzip the binary payload and pass it on as one or more unzipped binary payloads. All of which can done more transaparently with Snaps but the requirement to rotate the image led me to the script so I thought I’d see how much I could do without writing to the disk (answer - some not all). 🙂 Python, Powershell Example_2023_03_24.slp (235.8 KB)1.1KViews1like0CommentsRe: SnapLogic and Azure Repos Integration
Hi @alok.tiwari, We have just started looking at this also, and will hopefully be migrating from GitHub across to Azure in the next few weeks and then integrate further with Azure Dev Ops. Happy to swap notes, currently awaiting Azure security changes from our Infrastructure team. Matt.4.3KViews1like0CommentsRe: Access file in Python script
Hi @ash42, I’ve not found a way to read from the Snaplogic project folder itself but if you have a groundplex you can write to a folder of your choosing using a binary file writer and the path file://// Once done you can pick them up using the method suggested by @bojanvelevski. I did something similar but now pass the raw binary data into the Python script snap (use case is unzip and rotate images), it is more involved but might also save you the io of a additional disk write. Happy to share if you’re interested. Matt.3.4KViews0likes1CommentRe: Pagination with HTTP Client
In similar scenarios when the endpoint does not return the necessary info to determine the the ‘next page’ request directly I have used snap.in.totalCount I think in your scenario, the value returned by the above multiplied by the _pageSize pipeline parameter would equate to the Offest value you require for the next ‘next page’ request. Matt.4.4KViews1like1Comment