Tutorial: Using the DocuSign eSignature REST API with the REST Snap Pack and OAuth 2.0
DocuSign’s latest REST API version is 2.1 and also supports OAuth 2.0 authentication (replacing the legacy X-DocuSign-Authentication header-based auth). DocuSign also provides free (no credit card neccessary) developer accounts to try out their API. First, sign up for a DocuSign developer account here: DocuSign Developer Account Next, in the DocuSign Apps and Keys page inside the DocuSign Settings application, you’ll see your account information, including API Username, API Account ID, and Account’s Base URI. Click on the blue “Add App & Integration Key” button: Provide an appropriate application name and click “Add”: On the next page, your Integration Key (also known as a “Client ID”) will be displayed. Under the “Authentication” section, ensure the “Authorization Code Grant” radio button is enabled for a “User Application”. Then click on “Add Secret Key”: IMPORTANT! Copy this secret key value immediately and paste it somewhere just for now - it will no longer be available to be used once you navigate away from this page. Further down, under “Additional settings”, click the “Add URI” button and paste in the following value: https://elastic.snaplogic.com/api/1/rest/admin/oauth2callback/rest Finally, click the “Save” button and you’ll return to the “Apps and Keys” section of the Developer admin console, where your application will now be visible. Next, log into SnapLogic’s Designer web application, create a new pipeline and drag a REST GET Snap to the empty canvas. Switch to the “Account” tab and click the “Add Account” button: Now it’s time to use the values from the DocuSign developer/integration web console - in another tab, click on the application you created within DocuSign and enter the following information: Since we are using the DocuSign Demo Environment, we’ll be using the demo-specific account-d.docusign.com domain - the -d suffix is reserved for demo use. Click the “Authorize” button towards the bottom to initiate the process to get an OAuth 2.0 access token. Your web browser will send a request to the OAuth2 Endpoint URL (the Auth endpoint) - you may be asked to login and possibly to approve the application. After doing so, the request will redirect back to the SnapLogic platform (the redirect_uri where a new token request to the “OAuth2 Token” URL (the token endpoint) will automatically take place in the background and an access- and refresh tokens will be acquired. If all goes well, the browser will navigate back to the SnapLogic Designer and your account will now have encrypted entries for the “Access-” and “Refresh token” fields, plus a timestamp value for the “Access token expiration” settings field. Click “Apply” to save the account and we can now test that the access token we received is valid by making requests to the DocuSign eSignature REST API: GET User Info The first request will be to the User Info endpoint - https://account-d.docusign.com/oauth/userinfo This endpoint will return environment data such as accountId , userId , and baseUrl - these are required to make any eSignature API call: For Each Account Since the response’s entity.accounts is an array, we can use the JSON splitter to operate on the account data that we will leverage in subsequent API calls: GET Account Info Finally, we can now make a call to the DocuSign eSignature REST API. After the Splitter add another REST GET Snap, configure it with the same account used previously, and leverage the ability to refer to the input data ( $base_uri , $account_id ) to build the request URL to the demo.docusign.net API domain: Validating and/or executing this Snap should result in a 200 OK response with a response body containing the metadata of the desired account, which can be further isolated with a Mapper mapping the $entity field to $ : And that should get your started. Consult the DocuSign REST API reference documentation to learn about other endpoints that can be used.Connecting to Marketo with the REST Snap Pack
While SnapLogic’s REST OAuth account supports only OAuth 2.0, it does not work with Marketo’s OAuth implementation. To work with Marketo, you must authenticate manually using the REST Get Snap. In this pipeline, we pass the credentials in as pipeline parameters. Note: This method does expose your credentials in the pipeline. Authorization To simplify the process, define the following pipeline parameters url: the REST API URL for your Marketo instance, like: https://xxx-xxx-xxx.mktorest.com clientID: The clientID for API access. clientKey: The client secret for API access. Add a REST Get Snap (labeled Marketo Login here) and configure as follows: For Service URL, toggle on the Expression button ( = ) and set the field to: _url + '/identity/oauth/token?grant_type=client_credentials&client_id=' + _clientID + '&client_secret=' +_clientKey Remove the input view. Validate the Snap and it will a return a response that contains an access_token and scope. In this example, we follow the REST Get with a Mapper Snap to map the token outside of the array. Using the Access Token In subsequent Snaps, we pass this token as a header, rather than a query parameter because it simplifies paged operations such as Get Lead Changes. Here’s an example of a simple call which does this. For Service URL, toggle on the Expression button ( = ) and set the field to: _url + '/rest/v1/activities/types.json' Under HTTP Header, set Key to Authorization and Value with the Expression button ( = ) toggled on to ‘Bearer ‘ + $accessToken Paged Operations When you get to more complex operations, such as getting lead changes, you need to make two API calls: the first creates a paging token, and the second uses the paging token typically with the paging mechanism enabled in our REST GET Snap. Get Paging Token In this REST Get Snap (renamed Get Paging Token for clarity) is where you specify the query parameters. For instance, if you want to get lead changes since a particular date, you’d pass that in via “sinceDateTime”. The example provided uses a literal string, but could be a pipeline parameter or ideally one of a Date objects formatted to match what Marketo expects. _url + '/rest/v1/activities/pagingtoken.json' Configure Paging Mechanism When calling Get Leads (via a REST GET Snap), a few things to bear in mind: You need to pass “nextPageToken” as a query parameter, along with the fields you want back. Ideally, the list of fields should be in a pipeline parameter because they appear twice in this configuration. The leads will be returned in $entity.result, which is an array. This field will not exist if there are no results, so you need to enable “Null safe” on a Splitter Snap after this REST Get. Paging expressions for the REST Get Snap are: Has next: $entity.moreResult == true Next URL: '%s/rest/v1/activities/leadchanges.json?nextPageToken=%s&fields=firstName,lastName'.sprintf( _url, $entity.nextPageToken ) API Throttling Marketo throttles API calls. Their documentation says “100 API calls in a 20 second window”. Since our REST Snap paging now includes an option to wait for X seconds or milliseconds between requests, use it whenever you are retrieving paginated results. Downloads Marketo REST.slp (14.7 KB)
SnapLogic REST API Design - Best Practices
This document provides a guide on best practices for REST API design within the SnapLogic Platform. The REST API design guidelines are a collection of API design patterns and principles that all API teams within an organisation should adhere to when developing APIs. The definition and implementation of API design guidelines are among the most influential drivers of an API strategy — fostering a consistent approach to the creation of an API platform across the enterprise. Author and contributors; Roberto Oliva and Chris WardTutorial: Using the DocuSign eSignature REST API with the REST Snap Pack and JWT Authentication
I previously wrote about integrating with the DocuSign eSignature API with the REST Snap Pack and OAuth 2.0 authentication, but for those that wish to control their access a bit more, DocuSign supports authenticating with JSON Web Tokens (JWTs) too. I followed the instructions from the “How to get an access token with JWT Grant authentication” DocuSign Developer portal. Setup As before, DocuSign makes it very easy to get started with their free Developer Account signup. Again, in the sidebar under “INTEGRATIONS”, select “Apps and Keys”. One that page is up, choose the “Add App & Integration Key” button and provide an application name (I chose “SnapLogic Community JWT Demo” this time). The “Integration Key” that is generated is also known as the Client ID and JWT will also refer to it as the "iss" value. You can keep the User Application selected as Authorization Code Grant but instead of adding a Secret Key, this time we’re going to be choosing a Service Integration: I wanted DocuSign to generate the public/private key pair for me, so I selected the “Generate RSA” button. A new dialog will open this the generated keys - you must copy these values and store them somewhere as this will be your only chance to do so: Finally, I set a throwaway Redirect URI, http://localhost (this isn’t really used - the goal here is to first authorized the app you created but JWT will take over from that point). Click “Save” to create the Integration App. Grant Consent The first thing you want to do is to get consent from a DocuSign user (it could be you) for this App to impersonate them (that’s what a Service account does). It follows this URI syntax: https://account-d.docusign.com/oauth/auth? response_type=code &scope=YOUR_REQUESTED_SCOPES &client_id=YOUR_INTEGRATION_KEY &state=YOUR_CUSTOM_STATE &redirect_uri=YOUR_REDIRECT_URI so I opened the following URL in my browser (substitute your client ID/integration key, state, scopes and redirect URI that you wish to use): https://account-d.docusign.com/oauth/auth?response_type=code&scope=signature%20impersonation&client_id=edec7e1e-e642-451b-8dfd-e7a211b23b40&state=throwaway&redirect_uri=http://localhost You’ll be asked to consent to the application: and then redirected to that throwaway localhost redirect URI (you don’t need to save anything from this response). Generating the JWT Now that consent has been granted, it’s time to move to JWT. This is where the JWT Snap Pack comes into play. It’s outside the scope of this topic to fully describe JWT, but in short it is a token value that encodes header and payload/body data that is signed with keys. The DocuSign Developer docs take you through how its constructed. The JWT Generate Snap will take care of generated the correct header ( "kid" is the Key Alias ID - more on that later, "alg" has to be "RS256" , and "typ":"JWT" is implied by the JWT spec). The main action is to generate the JWT Payload/Body that matches what DocuSign wants: { "iss": "5c2b8d7e-xxxx-xxxx-xxxx-cda8a50dd73f", "sub": "464f7988-xxxx-xxxx-xxxx-781ee556ab7a", "aud": "account-d.docusign.com", "iat": 1598383123, "exp": 1598390123, "scope": "signature impersonation" } and there are few ways to do that. You could build the above JSON object manually than use it directly in the “Custom Metadata” section, but I’ll show how to leverage the various fields in the JWT Generate Snap and the JWT Account. "aud" and "sub" stand for Audience and Subject respectively and can be set directly on the JWT Generate Snap: "sub" is the most difficult value to get - in fact, you have to look it up by using the API to call the /userinfo endpoint and that means using the OAuth 2.0 flow. Luckily, my first post described exactly how to do this - follow the setup instructions and see the “GET User Info” section. Note: "aud" must be list of strings (hence the array). It will be disregarded silently if it is not and your auth will fail. We will investigate a usability enhancement here. The "iat" value will be generated automatically (it defaults to “now” in Unix Epoch timestamp format). "iss" is the issuer, and this will be the Client ID/Integration Key created when you registered your App in the DocuSign Developer portal. You configure the "iss" and "exp" values by creating a JWT Account and leveraging the “JWT Issuer” and “Token TTL” fields respectively (the latter is added to the generated "iat" value). That leaves the "scopes" (which will always include impersonation and most DocuSign APIs want the signature scope too, so I’ve include them both, space-separated), and that can be directly configured on the “Custom Metadata” field in the JWT Generate Snap. All this configuration between the Snap and Account settings will be combined to form the Payload (you may also see a "nbf" payload field automatically added, which is “not before” and defaults to 120 seconds before "iat" and “Token ID” becomes "jti" ; DocuSign will ignore these): Finally, we need to configure the JWT Generate Snap’s Account for signing. This is where the public/private keypair saved earlier will be used. We’ll want to create a protected keystore for use by the account. There are a variety of ways to do this, but I’ll demonstrate with openssl . The first step is to combine the public and private keys together into one PEM file: Then we’ll use openssl to create a X.509 certificate: And then we’ll combine the certificate and the private key in the PEM file to create a PKCS 12 (.p12) file format: Upload the .p12 file to the Account’s “Key Store” field, also providing the password you used to protect it. You can use the suggest bubble on the “Key Alias” field to refer to your entry by name (this will be the "kid" value in the JWT Header). Decoding the Generated JWT The output of the JWT Generate Snap will be the JWT token under the "access_token" field (don’t confuse this with a DocuSign API access token, that comes later): You can copy this value and use jwt.io to see the decoded values: Exchanging the JWT for a DocuSign API Access Token Alright, it’s finally time to actually get an access token from DocuSign and start using their API! We are going to use the REST POST Snap to send a form-encoded request to the DocuSign token endpoint: The changes are minimal - the HTTP Entity is "grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer&assertion="+$access_token (using the JWT you just created) and you need to set the appropriate Content-Type HTTP Header value of application/x-www-form-urlencoded . The Service URL is https://account-d.docusign.com/oauth/token and note that this needs to match the Audience value you used earlier (it is also the Demo environment value - Production is different; see DocuSign’s docs linked above). If everything has been configured correctly, you’ll get an API access token: And like in the first post, you can use this token with the REST Snap Pack (e.g. GET), albeit this time setting it directly via the Authorization HTTP Header: and it should return data successfully: In the end, this pipeline looked simply like this: Cheers!
Invoking a SL pipeline with a REST call
Hi all, I am utterly new to Snaplogic and I need to be able to invoke.execute a pipeline with a REST call ie I need to expose a trigger in SL as a REST endpoint that can be invoked by a service running else where. Is this possible and if so how? Thanks in advance, AngusREST OAuth2 Account - 'Client Credentials' grant type
I’m working with an application that supports ‘Client Credentials’ and ‘Authorization Code’ grant types in the authorization request; however, doesn’t support refresh tokens. What is the value of selecting the ‘Client Credentials’ grant type in the REST OAuth2 Account? Does the REST OAuth2 Account make the authorization request and get the token when the first document enters the REST snap that’s using the account? What’s the purpose of the ‘Authorize’ button when the grant type is ‘Client Credentials’ since the token will expire at some point? Is it just to validate that the account has been configured correctly? I have the same questions if the grant type is ‘Authorization Code’ and the api doesn’t support refresh tokens.
REST Get Slow
Hey guys, I have built a simple REST Get process to grab some data from an API. There are about 100 URLs to call but it is taking ages. 100 URLs takes about 7 mins whereas in another software application the 100 calls takes about 25s! I have tried using the REST Get snap in a child pipeline and increasing the pool size but I’m still not getting anywhere the speed I would expect. Any suggestions?
Collecting Error Data
Submitted by @stodoroska from Interworks The pipeline Error.Routine is collecting the error data and depending on the Error.Log parameters, these errors are logged into SQL Server database table and/or JSON file is created at some FTP location. Error.Routine Pipeline Error.Log Pipeline Configuration Parameters are configured for the parent and child pipelines. If the parameter used in the Filter Snap is set to logEnabled==‘true’, we are logging the error data, but if the parameter is logEnabled==‘false’, we are not processing further error data. Sources: Error data that is coming from Snap error view Targets: SQL Server table and/or a JSON file in File share system; email Snaps used: REST Get, Filter, Mapper, Pipeline execute, Router, SQL Server execute, JSON Formatter, File writer, Union, E-mail sender Downloads Error.Routine.slp (7.3 KB) Error.Log.slp (14.9 KB)
SnapLogic Public APIs for Pipeline Runtime Execution Info
Created by @cstewart This pipeline uses the SnapLogic Public APIs to read the summary of executions, then does two things with them; first writes the line detail retrieved into a file, and concurrently, also writes a summary into a file, so you can see what the output contains. This pipeline utilizes the following techniques: Using the iteration feature of the REST Get Snap so that the multiple iterative requests required to retrieve the detail from the SnapLogic Public APIs are consumed. Dynamically configuring the URLs used with the expression language ternary operator Using an expression to get the org name (for use in the URLs) Putting multiple different format data into a single output stream, with an implication of order Configuration The REST Get Snap will need a REST Basic Auth account with credentials valid to make the request to the platform, and the result will only include the results for those executions which those credentials have authority to see. Sources: SnapLogic API Targets: CSV and JSON Files Snaps used: REST Get, JSON Splitter, Copy, Aggregate, Mapper, Union, JSON Formatter, CSV Formatter, File Writer Downloads GetRuntimeInfo.slp (19.8 KB)
