Recent Discussions
trace API and proxy calls
Hi ! I'm new to Snaplogic and I would like to trace all API and proxy calls in Datadog. Is there a way in SnapLogic to access a list that contains all API and proxy calls that have been made, along with their response codes? Additionally, in order to create a dashboard in Datadog, where can I find the necessary information in SnapLogic to retrieve this data? Thank you for the help !
JWT Configuration for SnapLogic Public API
This document details the process of configuring JWT authentication for the SnapLogic Public API using self-generated keys without the use of any third party JWT providers. It covers key generation, JWKS creation, SnapLogic configuration. 1. Key Generation and JWKS Creation 1.1 Setup the CMD Open CMD Mount the OpenSSL bin folder 1.2 Generate the Private Key Use the following command to generate a 2048-bit RSA private key in the PEM format. BASH openssl genpkey -algorithm RSA -out jwt_private_key.pem -pkeyopt rsa_keygen_bits:2048 Result:A file named jwt_private_key.pem will be created. This key must be kept secret and secure. 1.3 Convert to PKCS#8 Format The JWT generation requires the private key to be in the PKCS#8 format for proper decoding. So, convert the jwt_private_key.pem into PKCS8 format. BASH openssl pkcs8 -topk8 -in jwt_private_key.pem -out jwt_private_key_pkcs8.pem -nocrypt Result:A new file, jwt_private_key_pkcs8.pem, will be created. Use this key in your application for signing JWTs. 1.4 Extract the Public Key The public key is required for the JWKS document. BASH openssl rsa -in jwt_private_key_pkcs8.pem -pubout -out jwt_public_key_pkcs8.pem Result:A file named jwt_public_key.pem will be created. 1.5 Extract Public Key Components for JWKS: Extract the Modulus and Exponent from the CA-signed public key. These are the core components of your JWKS. BASH openssl rsa -pubin -in jwt_public_key_pkcs8.pem -text -noout The output will look like this:Public-Key: (2048 bit)Modulus: 00:d2:e3:23:2c:15:a6:5b:54:c1:89:f7:5f:41:bf:...Exponent: 65537 (0x10001) 2. JWKS Creation and JWT Endpoint Configuration 2.1. The below steps explain how to create the JWKS JSON within Snaplogic. 2.1.1 Create a new project sapce and a project "JWKS" or even an API with name "JWKS" - (This step is just for access control and the API policy to be applied only for this purpose) 2.1.2 Create the pipeline CreateJWKS 2.1.3 Update the Modulus and Exponent values in the mapper copied from the step 1.5 in the section Key Generation, JWKS Creation, and Certificate Signing. 2.1.4 Select the language as Python and replace the default script in the script snap with # Import the interface required by the Script snap. from com.snaplogic.scripting.language import ScriptHook import base64 import hashlib class TransformScript(ScriptHook): def __init__(self, input, output, error, log): self.input = input self.output = output self.error = error self.log = log # Helper function to convert an integer to a big-endian byte string # This is a manual implementation of int.to_bytes() for Python 2.7 def int_to_bytes(self, n): if n == 0: return '\x00' hex_string = "%x" % n if len(hex_string) % 2 == 1: hex_string = '0' + hex_string return hex_string.decode("hex") def execute(self): self.log.info("Executing Transform script") while self.input.hasNext(): try: inDoc = self.input.next() # Modulus conversion logic hex_input = inDoc['hex_string_field'] clean_hex_string = hex_input.replace('\n', '').replace(' ', '').replace(':', '') modulus_bytes = clean_hex_string.decode("hex") modulus_base64url = base64.urlsafe_b64encode(modulus_bytes).rstrip('=') # Exponent conversion logic exponent_input_str = inDoc['exponent_field'] import re match = re.search(r'^\d+', exponent_input_str) if match: exponent_int = int(match.group(0)) else: raise ValueError("Could not parse exponent value from string.") exponent_bytes = self.int_to_bytes(exponent_int) exponent_base64url = base64.urlsafe_b64encode(exponent_bytes).rstrip('=') # Dynamic Key ID (kid) generation logic # Concatenate the Base64url-encoded modulus and exponent jwk_string = modulus_base64url + exponent_base64url # Compute the SHA-256 hash kid_hash = hashlib.sha256(jwk_string).digest() # Base64url encode the hash to create the kid kid = base64.urlsafe_b64encode(kid_hash).rstrip('=') # Prepare the output document with all values outDoc = { 'modulus_base64url': modulus_base64url, 'exponent_base64url': exponent_base64url, 'kid': kid } self.output.write(inDoc, outDoc) except Exception as e: errDoc = { 'error' : str(e) } self.log.error("Error in python script: " + str(e)) self.error.write(errDoc) self.log.info("Script executed") def cleanup(self): self.log.info("Cleaning up") hook = TransformScript(input, output, error, log) 2.1.5 Replace the default value in the JSON generator with { "keys": [ { "kty": "RSA", "alg": "RS256", "kid": $kid, "use": "sig", "e": $exponent_base64url, "n": $modulus_base64url } ] } This will return us the JWKS JSON. 2.2. The below step creates the public endpoint for the JWKS JSON. The below steps can be done as a standalone API as well as a separate project for this JWKS authentication. 2.2.1 Create the pipeline getJWKS 2.2.2 Paste the JWKS generated in step 2.1.5 above in the JSON Generator: { "keys": [ { "kty": "RSA", "alg": "RS256", "kid": "vTfx70NbtVbarHnBetDHNqLXsWVr4Ue5oC32TFNSMlc", "use": "sig", "e": "AQAB", "n": "ANLjIywVpltUwYn3X0G_********_3JmpnSh419wDZC_8-Ts" } ] } 2.2.3 Follow the config as shown for JSON Formatter: 2.2.4 Create a Task named jwks.json and follow the task config as shown and copy the Ultra Task HTTP Endpoint: Select the Snaplex as Cloud, as the endpoint have to be truly public. 2.2.5 Create an API Policy - Anonymous Authenticator and key in the details as shown: 2.2.6 Create an API Policy - Authorize By Role and key in the details as shown: 3. SnapLogic JWT Configuration This step links SnapLogic to your JWKS. Configure Admin Manager: 3.1 In the SnapLogic Admin Manager, navigate to Authentication > JWT. 3.1.1 Issuer ID: Enter a unique identifier for your issuer. This can be a custom string. 3.1.2 JWKS Endpoint: Enter the full HTTPS URL where you have hosted the JWKS JSON file, HTTP Endpoint copied from step B.4 in the Section JWKS Creation and JWT Endpoint Configuration. 3.2 In the SnapLogic Admin Manager, navigate to Allowlists > CORS allowlist 3.2.1 Add domain: Key in the domain https://*.snaplogic.com in the Domain text box, click on Add Domain and click on Save. 4. JWT Generation and Structure The JWT must be created with a header that references your custom kid and a payload with claims that match SnapLogic's requirements. 4.1 Header: JSON { "alg": "RS256", "typ": "JWT", "kid": "use the key id generated in step 2.1.5 from the section JWKS Creation and JWT Endpoint Configuration"} 4.2 Payload: JSON { "iat": {{timestampIAT}}, "exp": {{timestampEXP}}, "sub": "youremail@yourcompany.com", "aud": "https://elastic.snaplogic.com/api/1/rest/public", "iss": "issuer id given in section 3.1.1.1", "org": "Your Snaplogic Org" } 4.3 Sign the JWT: Use the jwt_private_key_pkcs8.pem to sign the token with your application's JWT library. 4.4 Postman Pre-Request script to automatically generate epoch timestamps for iat and exp claims let now = new Date().getTime(); let iat = (now/1000) let futureTime = now + (3600 * 1000); let exp = (futureTime/1000) // Set the collection variable pm.collectionVariables.set("timestampIAT", iat); pm.collectionVariables.set("timestampEXP", exp);
Welcome to the Gold Star to the Winner Challenge - Halloween 2025 Edition! ⭐️
From time to time I send out to my team at SnapLogic fun pipeline building challenges that Expression Enthusiasts may enjoy solving. We have decided to open it up to the broader Snaplogic Community. The Gold Star to the Winner Challenge Halloween 2025 Edition is the spookiest challenge of the year. Your job will be to cast a powerful spell in the form of an expression to tame some monstrously messy data. As usual, this challenge is from a real world use case. It centers on schemalessly transforming ‘somewhat’ structured data into a perfectly structured, “OCD-approved” format. The Details: In the following dataset, there are two keys: “Name” and “Path”. The Trick is to craft an expression that can magically break apart the Path string into separate keys, numbering them sequentially (pathelement_1, pathelement_2, etc.).For example: a path with 3 elements in it would transform to 3 json keys:Input JSON: { “Path”:“my drive/matt/customers” } Output JSON: { “pathelement_1: “my drive”, “pathelement_2": “matt”, “pathelement_3": “customers” } Here’s the raw input to be put in a JSON Generator: [{"Name":"Fred","Path":"spooky/graveyard/tombstones/fog/cackles/witches/brewing/potions/spells/hauntedhouse.jpg"},{"Name":"Wilma","Path":"kids/yard ornaments/ghosts/goblins/monsters/jack o lantern/leaves/cocoa/chill/candysacks/excitement/pumpkins/tricks/treats.png"},{"Name":"Pebbles","Path":"shadows/bats/moonlight/screams/night/costumes/party.mp4"},{"Name":"Dino","Path":"creepy/cornfields/scarecrows/spiders/webs.gif"}] And the expected output: [{"pathelement_1":"spooky","pathelement_2":"graveyard","pathelement_3":"tombstones","pathelement_4":"fog","pathelement_5":"cackles","pathelement_6":"witches","pathelement_7":"brewing","pathelement_8":"potions","pathelement_9":"spells","pathelement_10":"hauntedhouse.jpg","Name":"Fred"},{"pathelement_1":"kids","pathelement_2":"yard ornaments","pathelement_3":"ghosts","pathelement_4":"goblins","pathelement_5":"monsters","pathelement_6":"jack o lantern","pathelement_7":"leaves","pathelement_8":"cocoa","pathelement_9":"chill","pathelement_10":"candysacks","pathelement_11":"excitement","pathelement_12":"pumpkins","pathelement_13":"tricks","pathelement_14":"treats.png","Name":"Wilma"},{"pathelement_1":"shadows","pathelement_2":"bats","pathelement_3":"moonlight","pathelement_4":"screams","pathelement_5":"night","pathelement_6":"costumes","pathelement_7":"party.mp4","Name":"Pebbles"},{"pathelement_1":"creepy","pathelement_2":"cornfields","pathelement_3":"scarecrows","pathelement_4":"spiders","pathelement_5":"webs.gif","Name":"Dino"}] Solution approaches: There are many ways to skin this cat; highlighting the flexibility of the SnapLogic platform. My solution contains a single expression in a mapper. Others (the purists) have solved this by configuring and connecting many transform Snaps. All solutions are good as long as the solutions matches the above expected output and is done in a completely schemaless way. The Prize: The winner will receive recognition in the form of SnapLogic Swag (👕🥤🍾 🎁...). The rules: To keep the playing field level, send solutions directly to me via email (msager@snaplogic.com) and attach your pipeline .slp file. (i.e. we don't want to give solutions out on this post for others to see) Contest ends on 10/31/2025 Good Luck to All! I look forward to seeing your solutions.
Filter in map after aggregate & group by
Hi, I am using an Aggregate step with a Group By and I am trying to get a value based on another object's value. e.g. in the below example, I'd like to return the last_updated value where status = complete i.e. "2025-01-01" I tried this but it just returns true/false: $status == "Complete" ? $last_updated : null ``` [ { "status": "Complete", "last_updated": "2025-01-01" }, { "status": "Pending", "last_updated": "2025-05-01" } ] ``` Any help would be much appreciated! Thanks
