REST Post Multipart Form-Data + File Upload Issue
I’m having trouble correctly formatting multipart form-data to be sent as the body of a REST Post request along with a single file upload. I’ve managed to get the Post request to correctly send in Postman with the required key/value pairs and a test file: However, when attempting to replicate this in SnapLogic I am unsuccessful. I’ve managed to correctly map the key/value pairs in a mapper, and have created the following JSON in a JSON generator using my mapped values: I’ve used my “entity” object as the HTTP entity in my Post request, and pointed to a local test file for upload in my REST Post snap: Is my approach to compiling the form-data correct, or no?
401 Error using Https in API REST GET call
In house API call that works well in all browsers and postman returns “REST API service endpoint returned error result: status code = 401, reason phrase = Unauthorized, refer to the error_entity field in the error view document for more details” when using the REST snap The following was tried and the issue persist -Create account from REST GET similar to what is used to access API from browser (NTLM) -Trust all certificates settings checked on REST snap -‘Keep-alive’ connection added to Header settings -Cookie details added to Header settings Does anyone have any suggestion to assist in getting this error fixed
Connecting to Marketo with the REST Snap Pack
While SnapLogic’s REST OAuth account supports only OAuth 2.0, it does not work with Marketo’s OAuth implementation. To work with Marketo, you must authenticate manually using the REST Get Snap. In this pipeline, we pass the credentials in as pipeline parameters. Note: This method does expose your credentials in the pipeline. Authorization To simplify the process, define the following pipeline parameters url: the REST API URL for your Marketo instance, like: https://xxx-xxx-xxx.mktorest.com clientID: The clientID for API access. clientKey: The client secret for API access. Add a REST Get Snap (labeled Marketo Login here) and configure as follows: For Service URL, toggle on the Expression button ( = ) and set the field to: _url + '/identity/oauth/token?grant_type=client_credentials&client_id=' + _clientID + '&client_secret=' +_clientKey Remove the input view. Validate the Snap and it will a return a response that contains an access_token and scope. In this example, we follow the REST Get with a Mapper Snap to map the token outside of the array. Using the Access Token In subsequent Snaps, we pass this token as a header, rather than a query parameter because it simplifies paged operations such as Get Lead Changes. Here’s an example of a simple call which does this. For Service URL, toggle on the Expression button ( = ) and set the field to: _url + '/rest/v1/activities/types.json' Under HTTP Header, set Key to Authorization and Value with the Expression button ( = ) toggled on to ‘Bearer ‘ + $accessToken Paged Operations When you get to more complex operations, such as getting lead changes, you need to make two API calls: the first creates a paging token, and the second uses the paging token typically with the paging mechanism enabled in our REST GET Snap. Get Paging Token In this REST Get Snap (renamed Get Paging Token for clarity) is where you specify the query parameters. For instance, if you want to get lead changes since a particular date, you’d pass that in via “sinceDateTime”. The example provided uses a literal string, but could be a pipeline parameter or ideally one of a Date objects formatted to match what Marketo expects. _url + '/rest/v1/activities/pagingtoken.json' Configure Paging Mechanism When calling Get Leads (via a REST GET Snap), a few things to bear in mind: You need to pass “nextPageToken” as a query parameter, along with the fields you want back. Ideally, the list of fields should be in a pipeline parameter because they appear twice in this configuration. The leads will be returned in $entity.result, which is an array. This field will not exist if there are no results, so you need to enable “Null safe” on a Splitter Snap after this REST Get. Paging expressions for the REST Get Snap are: Has next: $entity.moreResult == true Next URL: '%s/rest/v1/activities/leadchanges.json?nextPageToken=%s&fields=firstName,lastName'.sprintf( _url, $entity.nextPageToken ) API Throttling Marketo throttles API calls. Their documentation says “100 API calls in a 20 second window”. Since our REST Snap paging now includes an option to wait for X seconds or milliseconds between requests, use it whenever you are retrieving paginated results. Downloads Marketo REST.slp (14.7 KB)
Pagination in REST API
A very good explanation of how to implement pagination in REST API in this below blog post: SnapLogic – 4 Feb 16 REST GET and the SnapLogic Public APIs for Pipeline Executions As a part of a wider analytics project I’m working on, analyzing runtime information from the SnapLogic platform, I chose ... When I try to implement the same(I am extracting data from survey monkey api), I have noticed that when the “Next URL” is executing it is ignoring the query parameters provided.As shown in below image, when service URL is executing it’s getting right records considering all input query parameters i.e total:3043 but when “Next URL” is hitting it is extracting all records i.e. total : 61896 How to include query parameters in “Next URL” to get required data?REST OAuth2 Account - 'Client Credentials' grant type
I’m working with an application that supports ‘Client Credentials’ and ‘Authorization Code’ grant types in the authorization request; however, doesn’t support refresh tokens. What is the value of selecting the ‘Client Credentials’ grant type in the REST OAuth2 Account? Does the REST OAuth2 Account make the authorization request and get the token when the first document enters the REST snap that’s using the account? What’s the purpose of the ‘Authorize’ button when the grant type is ‘Client Credentials’ since the token will expire at some point? Is it just to validate that the account has been configured correctly? I have the same questions if the grant type is ‘Authorization Code’ and the api doesn’t support refresh tokens.
Tutorial: Using the DocuSign eSignature REST API with the REST Snap Pack and JWT Authentication
I previously wrote about integrating with the DocuSign eSignature API with the REST Snap Pack and OAuth 2.0 authentication, but for those that wish to control their access a bit more, DocuSign supports authenticating with JSON Web Tokens (JWTs) too. I followed the instructions from the “How to get an access token with JWT Grant authentication” DocuSign Developer portal. Setup As before, DocuSign makes it very easy to get started with their free Developer Account signup. Again, in the sidebar under “INTEGRATIONS”, select “Apps and Keys”. One that page is up, choose the “Add App & Integration Key” button and provide an application name (I chose “SnapLogic Community JWT Demo” this time). The “Integration Key” that is generated is also known as the Client ID and JWT will also refer to it as the "iss" value. You can keep the User Application selected as Authorization Code Grant but instead of adding a Secret Key, this time we’re going to be choosing a Service Integration: I wanted DocuSign to generate the public/private key pair for me, so I selected the “Generate RSA” button. A new dialog will open this the generated keys - you must copy these values and store them somewhere as this will be your only chance to do so: Finally, I set a throwaway Redirect URI, http://localhost (this isn’t really used - the goal here is to first authorized the app you created but JWT will take over from that point). Click “Save” to create the Integration App. Grant Consent The first thing you want to do is to get consent from a DocuSign user (it could be you) for this App to impersonate them (that’s what a Service account does). It follows this URI syntax: https://account-d.docusign.com/oauth/auth? response_type=code &scope=YOUR_REQUESTED_SCOPES &client_id=YOUR_INTEGRATION_KEY &state=YOUR_CUSTOM_STATE &redirect_uri=YOUR_REDIRECT_URI so I opened the following URL in my browser (substitute your client ID/integration key, state, scopes and redirect URI that you wish to use): https://account-d.docusign.com/oauth/auth?response_type=code&scope=signature%20impersonation&client_id=edec7e1e-e642-451b-8dfd-e7a211b23b40&state=throwaway&redirect_uri=http://localhost You’ll be asked to consent to the application: and then redirected to that throwaway localhost redirect URI (you don’t need to save anything from this response). Generating the JWT Now that consent has been granted, it’s time to move to JWT. This is where the JWT Snap Pack comes into play. It’s outside the scope of this topic to fully describe JWT, but in short it is a token value that encodes header and payload/body data that is signed with keys. The DocuSign Developer docs take you through how its constructed. The JWT Generate Snap will take care of generated the correct header ( "kid" is the Key Alias ID - more on that later, "alg" has to be "RS256" , and "typ":"JWT" is implied by the JWT spec). The main action is to generate the JWT Payload/Body that matches what DocuSign wants: { "iss": "5c2b8d7e-xxxx-xxxx-xxxx-cda8a50dd73f", "sub": "464f7988-xxxx-xxxx-xxxx-781ee556ab7a", "aud": "account-d.docusign.com", "iat": 1598383123, "exp": 1598390123, "scope": "signature impersonation" } and there are few ways to do that. You could build the above JSON object manually than use it directly in the “Custom Metadata” section, but I’ll show how to leverage the various fields in the JWT Generate Snap and the JWT Account. "aud" and "sub" stand for Audience and Subject respectively and can be set directly on the JWT Generate Snap: "sub" is the most difficult value to get - in fact, you have to look it up by using the API to call the /userinfo endpoint and that means using the OAuth 2.0 flow. Luckily, my first post described exactly how to do this - follow the setup instructions and see the “GET User Info” section. Note: "aud" must be list of strings (hence the array). It will be disregarded silently if it is not and your auth will fail. We will investigate a usability enhancement here. The "iat" value will be generated automatically (it defaults to “now” in Unix Epoch timestamp format). "iss" is the issuer, and this will be the Client ID/Integration Key created when you registered your App in the DocuSign Developer portal. You configure the "iss" and "exp" values by creating a JWT Account and leveraging the “JWT Issuer” and “Token TTL” fields respectively (the latter is added to the generated "iat" value). That leaves the "scopes" (which will always include impersonation and most DocuSign APIs want the signature scope too, so I’ve include them both, space-separated), and that can be directly configured on the “Custom Metadata” field in the JWT Generate Snap. All this configuration between the Snap and Account settings will be combined to form the Payload (you may also see a "nbf" payload field automatically added, which is “not before” and defaults to 120 seconds before "iat" and “Token ID” becomes "jti" ; DocuSign will ignore these): Finally, we need to configure the JWT Generate Snap’s Account for signing. This is where the public/private keypair saved earlier will be used. We’ll want to create a protected keystore for use by the account. There are a variety of ways to do this, but I’ll demonstrate with openssl . The first step is to combine the public and private keys together into one PEM file: Then we’ll use openssl to create a X.509 certificate: And then we’ll combine the certificate and the private key in the PEM file to create a PKCS 12 (.p12) file format: Upload the .p12 file to the Account’s “Key Store” field, also providing the password you used to protect it. You can use the suggest bubble on the “Key Alias” field to refer to your entry by name (this will be the "kid" value in the JWT Header). Decoding the Generated JWT The output of the JWT Generate Snap will be the JWT token under the "access_token" field (don’t confuse this with a DocuSign API access token, that comes later): You can copy this value and use jwt.io to see the decoded values: Exchanging the JWT for a DocuSign API Access Token Alright, it’s finally time to actually get an access token from DocuSign and start using their API! We are going to use the REST POST Snap to send a form-encoded request to the DocuSign token endpoint: The changes are minimal - the HTTP Entity is "grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer&assertion="+$access_token (using the JWT you just created) and you need to set the appropriate Content-Type HTTP Header value of application/x-www-form-urlencoded . The Service URL is https://account-d.docusign.com/oauth/token and note that this needs to match the Audience value you used earlier (it is also the Demo environment value - Production is different; see DocuSign’s docs linked above). If everything has been configured correctly, you’ll get an API access token: And like in the first post, you can use this token with the REST Snap Pack (e.g. GET), albeit this time setting it directly via the Authorization HTTP Header: and it should return data successfully: In the end, this pipeline looked simply like this: Cheers!
