Recent Discussions
Preview not available for Flow snaps
Certain flow snaps like “Filter”, “Head”, “Tail” when used in a complex pipeline doesn’t generate a preview while validating the entire pipeline however these generate a preview if the pipeline is simple. For instance, if I’m trying to read data from an excel file with 100k+ records and use either ‘filter’ or ‘head/tail’, I can see the preview when the pipeline has been validated but it doesn’t generate one when the pipeline is huge. Any specific reason behind this? I would want to see the preview as the pipeline that I’m currently developing is a complex one (multiple snaps prior to “flow” snaps), thereafter it needs a ‘filter + head/tail’ (i.e. flow snaps) and then it will again require a bunch of other snaps after those ‘filter + head/tail’ making it complex again. Help on this matter would be highly appreciated. Regards, DarshSolveddarshthakkar4 years agoValued Contributor31KViews0likes13CommentsMaven Install Authentication Error
Hello. Trying to do some development and cant seem to get out of the gates. I have tried this on multiple networks so it does not appear to be network related. When run “ mvn clean install ” in my newly created workspace, after successfully running “ mvn org.apache.maven.plugins:maven-archetype-plugin:2.4:generate -DarchetypeCatalog=http://maven.clouddev.snaplogic.com:8080/nexus/content/repositories/master ” I get the following error. Failed to read artifact descriptor for com.snaplogic:jutils:jar:4.22: Could not transfer artifact com.snaplogic:jutils:pom:4.22 from/to github (https://maven.pkg.github.com/SnapLogic/Tectonic): Authentication failed for https://maven.pkg.github.com/SnapLogic/Tectonic/com/snaplogic/jutils/4.22/jutils-4.22.pom 401 Unauthorized Thoughts?ronmwhite6 years agoNew Contributor II12KViews0likes19CommentsMerge files with varying column numbers
Continuing the discussion from Csv file with varying number of columns in each row: Here is what the file looked like when i started: EEID|55555|xxxxxxxxx|MICHAEL|C|xxxxx|M|06/21/xxxx OFFR|55555|01/01/2023-55|OE|01/01/2023|01/01/2023|01/01/2023|12/31/2023|02/06/2023 05:00:00.000000 AM ELIG|55555|01/01/2023-55|CDHU02500L|BLUE BASIC|EEONLY|EMPLOYEE ONLY|Y|103.28|511.62|Y|Y|Y|Y|Y||| COVG|55555|OE|01/01/2023|||||||01/01/2023|||||Y|06|UNKNOWN|02/06/2023 05:00:00.000000 AM|01/01/2023-55 EEID|7777777|xxxxxxxxx|KEITH|E|xxxxxxxx|M|11/11/xxxx OFFR|7777777|01/01/2023-55|OE|01/01/2023|01/01/2023|01/01/2023|12/31/2023|02/06/2023 05:00:00.000000 AM ELIG|7777777|01/01/2023-55|CDHU02500L|BLUE BASIC|EEONLY|EMPLOYEE ONLY|Y|203.28|411.62|Y|Y|Y|Y|Y||| COVG|7777777|OE|01/01/2023|||||||01/01/2023|||||Y|06|UNKNOWN|02/06/2023 05:00:00.000000 AM|01/01/2023-55 Now it is a set of files with the rows above separated like this. The solution i showed worked fine. I am not able to put the files back togther into one file now because of the same error i received and displayed on the screen above. EEID|55555|xxxxxxxxx|MICHAEL|C|xxxxx|M|06/21/xxxx EEID|7777777|xxxxxxxxx|KEITH|E|xxxxxxxx|M|11/11/xxxx OFFR|55555|01/01/2023-55|OE|01/01/2023|01/01/2023|01/01/2023|12/31/2023|02/06/2023 05:00:00.000000 AM OFFR|7777777|01/01/2023-55|OE|01/01/2023|01/01/2023|01/01/2023|12/31/2023|02/06/2023 05:00:00.000000 AM ELIG|55555|01/01/2023-55|CDHU02500L|BLUE BASIC|EEONLY|EMPLOYEE ONLY|Y|103.28|511.62|Y|Y|Y|Y|Y||| ELIG|7777777|01/01/2023-55|CDHU02500L|BLUE BASIC|EEONLY|EMPLOYEE ONLY|Y|203.28|411.62|Y|Y|Y|Y|Y||| COVG|55555|OE|01/01/2023|||||||01/01/2023|||||Y|06|UNKNOWN|02/06/2023 05:00:00.000000 AM|01/01/2023-55 COVG|7777777|OE|01/01/2023|||||||01/01/2023|||||Y|06|UNKNOWN|02/06/2023 05:00:00.000000 AM|01/01/2023-55Solvedheidi_andrew3 years agoContributor12KViews0likes3CommentsHow to render distinct elements only in an array with duplicates
I have a JSON document that has inside an array with duplicate values such as [1,1,3,3,6,1,2,1,2,100]. I am trying to transform/map it from source into target by returning only the distinct values such as [1,3,6,100]. How can I achieve that using a Mapper snap or if any other snap. For example when using jsonPath() function to specify a part of JSON document that has an array, is there a function similar to .unique() that could be attached to return only unique array elements? The Unique snap works only on entire documents/rows not array elements within a document.mohamadelmardin9 years agoNew Contributor III11KViews0likes9CommentsIssue with Arrays in Downstream Mappers after a Custom Snap Transformation
We built a custom snap that performs a transformation. You will notice in the output of the custom snap in the screenshot below that “medicalEnrollment” is a json array. In a downstream mapper, we attempted a transformation on this array. We have tried (1) using the sl.ensureArray() method and (2) another downstream mapper performing a transformation in the array. However, we notice that in both the #1 and #2 attempts the mappers do not detect the array, thus causing issues with the transformation. Below is a snippet of the preview of what we expect vs actual preview. We also notice that chaining a Json Formatter and a Json Parser right after the custom snap does indeed makes the mappers work correctly. Pattern shown below: So my question is, is there something hidden in the JSON parser snap that allows strong typing of the array that we are missing? Anything in between? @robin , Tagging you since I was told you might be able to help out. Thanks!Solvedcclaudio4 years agoNew Contributor II9.4KViews0likes20CommentsHow to get the file name(s) from a Multi File Reader
I have a Multi File Reader reading a series of S3 files using a wildcard, and writing the data to Snowflake. There is a Mapper in between. Functionally, everything is working as expected. I’d like to get the name of the file in which the data was read, and write it to a Meta_FileName column. How do I retrieve the file name from the Multi File Reader? I’m sort of assuming it’s an expression to be added in the Mapper but not sure. TIA!mxpeskir7 years agoNew Contributor9.2KViews0likes14CommentsREST OAuth2 Account - 'Client Credentials' grant type
I’m working with an application that supports ‘Client Credentials’ and ‘Authorization Code’ grant types in the authorization request; however, doesn’t support refresh tokens. What is the value of selecting the ‘Client Credentials’ grant type in the REST OAuth2 Account? Does the REST OAuth2 Account make the authorization request and get the token when the first document enters the REST snap that’s using the account? What’s the purpose of the ‘Authorize’ button when the grant type is ‘Client Credentials’ since the token will expire at some point? Is it just to validate that the account has been configured correctly? I have the same questions if the grant type is ‘Authorization Code’ and the api doesn’t support refresh tokens.PSAmmirata6 years agoEmployee9.2KViews1like17CommentsCsv file with varying number of columns in each row
my csv file has several rows for each person on the file. The specific rows have different number of columns in each row. CSV file reader is having trouble with it. Is this impossible to read for this snap?Solvedheidi_andrew3 years agoContributor8.4KViews0likes17CommentsTutorial: Using the DocuSign eSignature REST API with the REST Snap Pack and JWT Authentication
I previously wrote about integrating with the DocuSign eSignature API with the REST Snap Pack and OAuth 2.0 authentication, but for those that wish to control their access a bit more, DocuSign supports authenticating with JSON Web Tokens (JWTs) too. I followed the instructions from the “How to get an access token with JWT Grant authentication” DocuSign Developer portal. Setup As before, DocuSign makes it very easy to get started with their free Developer Account signup. Again, in the sidebar under “INTEGRATIONS”, select “Apps and Keys”. One that page is up, choose the “Add App & Integration Key” button and provide an application name (I chose “SnapLogic Community JWT Demo” this time). The “Integration Key” that is generated is also known as the Client ID and JWT will also refer to it as the "iss" value. You can keep the User Application selected as Authorization Code Grant but instead of adding a Secret Key, this time we’re going to be choosing a Service Integration: I wanted DocuSign to generate the public/private key pair for me, so I selected the “Generate RSA” button. A new dialog will open this the generated keys - you must copy these values and store them somewhere as this will be your only chance to do so: Finally, I set a throwaway Redirect URI, http://localhost (this isn’t really used - the goal here is to first authorized the app you created but JWT will take over from that point). Click “Save” to create the Integration App. Grant Consent The first thing you want to do is to get consent from a DocuSign user (it could be you) for this App to impersonate them (that’s what a Service account does). It follows this URI syntax: https://account-d.docusign.com/oauth/auth? response_type=code &scope=YOUR_REQUESTED_SCOPES &client_id=YOUR_INTEGRATION_KEY &state=YOUR_CUSTOM_STATE &redirect_uri=YOUR_REDIRECT_URI so I opened the following URL in my browser (substitute your client ID/integration key, state, scopes and redirect URI that you wish to use): https://account-d.docusign.com/oauth/auth?response_type=code&scope=signature%20impersonation&client_id=edec7e1e-e642-451b-8dfd-e7a211b23b40&state=throwaway&redirect_uri=http://localhost You’ll be asked to consent to the application: and then redirected to that throwaway localhost redirect URI (you don’t need to save anything from this response). Generating the JWT Now that consent has been granted, it’s time to move to JWT. This is where the JWT Snap Pack comes into play. It’s outside the scope of this topic to fully describe JWT, but in short it is a token value that encodes header and payload/body data that is signed with keys. The DocuSign Developer docs take you through how its constructed. The JWT Generate Snap will take care of generated the correct header ( "kid" is the Key Alias ID - more on that later, "alg" has to be "RS256" , and "typ":"JWT" is implied by the JWT spec). The main action is to generate the JWT Payload/Body that matches what DocuSign wants: { "iss": "5c2b8d7e-xxxx-xxxx-xxxx-cda8a50dd73f", "sub": "464f7988-xxxx-xxxx-xxxx-781ee556ab7a", "aud": "account-d.docusign.com", "iat": 1598383123, "exp": 1598390123, "scope": "signature impersonation" } and there are few ways to do that. You could build the above JSON object manually than use it directly in the “Custom Metadata” section, but I’ll show how to leverage the various fields in the JWT Generate Snap and the JWT Account. "aud" and "sub" stand for Audience and Subject respectively and can be set directly on the JWT Generate Snap: "sub" is the most difficult value to get - in fact, you have to look it up by using the API to call the /userinfo endpoint and that means using the OAuth 2.0 flow. Luckily, my first post described exactly how to do this - follow the setup instructions and see the “GET User Info” section. Note: "aud" must be list of strings (hence the array). It will be disregarded silently if it is not and your auth will fail. We will investigate a usability enhancement here. The "iat" value will be generated automatically (it defaults to “now” in Unix Epoch timestamp format). "iss" is the issuer, and this will be the Client ID/Integration Key created when you registered your App in the DocuSign Developer portal. You configure the "iss" and "exp" values by creating a JWT Account and leveraging the “JWT Issuer” and “Token TTL” fields respectively (the latter is added to the generated "iat" value). That leaves the "scopes" (which will always include impersonation and most DocuSign APIs want the signature scope too, so I’ve include them both, space-separated), and that can be directly configured on the “Custom Metadata” field in the JWT Generate Snap. All this configuration between the Snap and Account settings will be combined to form the Payload (you may also see a "nbf" payload field automatically added, which is “not before” and defaults to 120 seconds before "iat" and “Token ID” becomes "jti" ; DocuSign will ignore these): Finally, we need to configure the JWT Generate Snap’s Account for signing. This is where the public/private keypair saved earlier will be used. We’ll want to create a protected keystore for use by the account. There are a variety of ways to do this, but I’ll demonstrate with openssl . The first step is to combine the public and private keys together into one PEM file: Then we’ll use openssl to create a X.509 certificate: And then we’ll combine the certificate and the private key in the PEM file to create a PKCS 12 (.p12) file format: Upload the .p12 file to the Account’s “Key Store” field, also providing the password you used to protect it. You can use the suggest bubble on the “Key Alias” field to refer to your entry by name (this will be the "kid" value in the JWT Header). Decoding the Generated JWT The output of the JWT Generate Snap will be the JWT token under the "access_token" field (don’t confuse this with a DocuSign API access token, that comes later): You can copy this value and use jwt.io to see the decoded values: Exchanging the JWT for a DocuSign API Access Token Alright, it’s finally time to actually get an access token from DocuSign and start using their API! We are going to use the REST POST Snap to send a form-encoded request to the DocuSign token endpoint: The changes are minimal - the HTTP Entity is "grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer&assertion="+$access_token (using the JWT you just created) and you need to set the appropriate Content-Type HTTP Header value of application/x-www-form-urlencoded . The Service URL is https://account-d.docusign.com/oauth/token and note that this needs to match the Audience value you used earlier (it is also the Demo environment value - Production is different; see DocuSign’s docs linked above). If everything has been configured correctly, you’ll get an API access token: And like in the first post, you can use this token with the REST Snap Pack (e.g. GET), albeit this time setting it directly via the Authorization HTTP Header: and it should return data successfully: In the end, this pipeline looked simply like this: Cheers!