HTTP Snap Query parameters issue
So, I have a strange issue with the HTTP snap that I have huge trouble solving. When validating the pipeline I get an error saying query param is not allowed for every query parameter provided in the query parameter fields. But when I copy the request string from HTTP Snap debug and paste it into Postman, the request works as expected. I've tested it both using query parameter fields and by manually mapping the URI with encoded values where appropriate. The request string from debug looks like this (Identifiers have been replaced with *****): /rest/organizationalEntityShareStatistics?q=organizationalEntity&organizationalEntity=urn%3Ali%3Aorganization%3A********&timeIntervals.timeGranularityType=DAY&timeIntervals.timeRange.start=1715844351774&timeIntervals.timeRange.end=1747380351774&ugcPosts[0]=urn%3Ali%3AugcPost%3A***************&ugcPosts[1]=urn%3Ali%3AugcPost%3A**************&ugcPosts[2]=urn%3Ali%3AugcPost%3A**************&ugcPosts[3]=urn%3Ali%3AugcPost%3A************** Does anyone have a clue what the issue may be or what I should do to troubleshoot it further? Thanks!
Performance Optimization: Use Static Lookups Instead of Joins
Hi! I wanted to share a powerful and lightweight approach I recently implemented for using static reference data in pipelines—without needing memory-intensive joins or separate file/database reads during runtime. The Challenge In typical scenarios, we handle static or reference data (like lookup tables or code descriptions) by: Reading it from a file or database Performing a Join Snap to enrich the main data stream While effective, joins: Can be memory-heavy, especially with large datasets Add complexity to your pipeline Require both sources to be aligned in structure and timing The New Approach Instead of performing a join, we can: Store static reference data as a JSON file in SnapLogic’s SLDB Load this JSON file in an Expression Library Use filter/map function in your pipeline expressions to fetch data from JSON based on a key No joins. No file readers. Just fast in-memory lookups! Example Sample JSON file (staticData,json) [ { "code": "A1", "desc": "Alpha" }, { "code": "B2", "desc": "Beta" }, { "code": "C3", "desc": "Gamma" } ] Define in Pipeline: Usage in Pipeline: lib.static.filter(x =>x.code == $code_from_source).length > 0 ? lib.static.filter(x =>x.code == $code_from_source)[0].desc : "Unknown" This setup allows you to quickly enrich your data using a simple expression, and the same logic can be reused across multiple pipelines via the library. Benefits Faster: No join processing overhead Simpler pipelines: Fewer snaps and data dependencies Reusable: One JSON file + one function = many pipelines Memory-efficient: Especially helpful when Snaplex memory is a constraint Things to Consider SLDB file size limit: The JSON file stored in SLDB must be under 100MB (SnapLogic’s file size limit for SLDB uploads). Data updates: If your reference data changes frequently (e.g., weekly/monthly), you’ll need to build a separate job or pipeline to overwrite the SLDB file. Search performance: The filter() method checks each item one by one, which can be slow if your JSON has a lot of records. For faster lookups, consider converting the data into a key-value map. Governance: SLDB files have limited access control compared to databases. Ensure your team is aligned on ownership and update responsibility. Maintainability: JSON logic is hardcoded, so changes to structure or logic require modifying the expression library and possibly redeploying affected pipelines. I’ve found this approach especially useful for small to medium-sized static datasets where performance, simplicity, and reusability are key. If you're looking to reduce joins and streamline your pipelines, I highly recommend giving this method a try. To make it easier, I’ve attached a sample pipeline, JSON lookup file, and input CSV so you can see the setup in action. Feel free to explore, adapt, and let me know how it works for you!Get MIN (oldest) or MAX (most recent) date from array of dates
Hi, I first do an SQL Server - Execute and get this weird {_snaptype_localdatetime: "..."} for the CreationDate and LastUpdate fields. Then, using a Mapper snap I do toString() to just get the date fields in string form. Then, I do a Group By on some fields and add another Mapper snap where I have this: Here, I am trying to get the MAX and MIN dates from the array. This is what the whole pipeline looks like: How can I get the oldest and the most recent dates from the array? Thank you very much in advance! Kind regards, Adam
HTTP Client snap issue encoding diacritical marks/accents
Hello all! Hoping for some helpful suggestions or insight regarding an issue we're facing with the new HTTP Client snap. We're sending a POST request with a JSON body. Within the body are street addresses, and some addresses contain diacritical marks/accents. An example would be "Houssières". We're supplying the body ($body) from an input Mapper into the snap as a raw entity, with the Content-Type header set to application/json: I can't supply the full input body as it contains PII unfortunately, but the street address is supplied within the body like so. You can see the accent mark in question: On debug, the snap parses the input body into a string labeled "requestString" within the response body, and within the string the accented character is shown as an unrecognized character: The encoding issue is causing the endpoint we're POSTing to to respond with a 400: So, is there some way I can combat this/force the snap to correctly encode similar strings within the request body? Maybe via JS within the raw entity in the snap, somehow? Anything I'm overlooking here that someone else has caught? For context this isn't an issue we've faced with the older REST POST snap, but we like the extended functionality of the HTTP Client snap, and would prefer to use it (if feasible). Thank you!
jsonpath in expression library
I would like to use a jsonpath in an expression library, but I am unsure of the syntax with parameters. custfield is a json array of custom field data and fieldname is a string This is what I attempted: { customField: (custfield,fieldname) => jsonPath($, "custfield[*]").find(f=>f.FieldName==fieldname) } What should the syntax be?Filtering Objects
Hello Experts, I have below Object input containing string values. Can someone please provide expression to get rid of duplicate values? [ { "ack": { "QueueURL": "https://www.google.com", "ReceiptHandle": "12345678", "content": "ABCDEFG" } }, { "ack": { "QueueURL": "https://www.google.com", "ReceiptHandle": "12345678", "content": "ABCDEFG" } }, { "ack": { "QueueURL": "https://www.google.com", "ReceiptHandle": "12345678", "content": "ABCDEFG" } } ] Thanks in Advance
