Designing and Running Pipelines

Discuss how to use the SnapLogic platform.

Forum Widgets

Recent Discussions

Pagination Logic Fails After Migrating from REST GET to HTTP Client Snap
Hello everyone, Three years ago, I developed a pipeline to extract data from ServiceNow and load it into Snowflake. As part of this, I implemented pagination logic to handle multi-page responses by checking for the presence of a "next" page and looping through until all data was retrieved. This job has been running successfully in production without any issues. Recently, we were advised by the Infrastructure team to replace the REST GET Snap with the HTTP Client Snap, as the former is being deprecated and is no longer recommended. I updated the pipeline accordingly, but the pagination logic that worked with REST GET is not functioning as expected with the HTTP Client Snap. The logic I used is as follows: Pagination → Has Next: isNaN($headers['link'].match(/",<([^;"]*)>;rel="next",/)) Override URI → Next URL: $headers['link'].match(/\",<([^;\"]*)>;rel=\"next\",/) ? $headers['link'].match(/\",<([^;\"]*)>;rel=\"next\",/)[1].replace(_servicenow_cloud_base_url, _servicenow_b2b_base_url) : null However, with the HTTP Client Snap, I’m encountering the following error: Error Message: Check the spelling of the property or, if the property is optional, use the get() method (e.g., $headers.get('link')) Reason: 'link' was not found while evaluating the sub-expression '$headers['link']' This exact logic works perfectly in the existing job using REST GET, with no changes to the properties. It seems the HTTP Client Snap is not recognizing or parsing the link header in the same way.
Solved
adityamohanty
New Contributor II
2 days ago
57Views
0likes
1Comment
Filter in map after aggregate & group by
Hi, I am using an Aggregate step with a Group By and I am trying to get a value based on another object's value. e.g. in the below example, I'd like to return the last_updated value where status = complete i.e. "2025-01-01" I tried this but it just returns true/false: $status == "Complete" ? $last_updated : null ``` [ { "status": "Complete", "last_updated": "2025-01-01" }, { "status": "Pending", "last_updated": "2025-05-01" } ] ``` Any help would be much appreciated! Thanks
Colossal8333
New Contributor II
5 days ago
49Views
0likes
3Comments
List column names from excel files
Hi I have multiple excel files, have to read all of them and list the Column names along with excel file names file1 column1 file1 column2 file1 column3 file2 column1
Solved
kumar25
New Contributor II
12 days ago
39Views
0likes
1Comment
No output preview using a specific groundplex
A simple JSON Generator snap will provide the output preview on one GroundPlex, and not provide the output preview on another GroundPlex. Setting all look the same. I'm guessing there is something obscure going on and need your help. Thank you very much
tony_hudnut
New Contributor
12 days ago
20Views
0likes
1Comment
Hi, Is there a way where we can add delay/wait of 3~5 seconds before every post call?
Hi, I have a requirement where i need to post (http client post call) a data by splitting it into multiple batches, like 100 records per batch. so, is there a way where we can add delay/wait of 3~5 seconds before every post call?
Solved
manjunathsinghs
New Contributor II
12 days ago
27Views
0likes
1Comment
Automating Untracked assets to GIT
Hi I am trying to understand if there’s a way to automate committing untracked assets to Git. Specifically, I’d like to know Is there any public API that allows adding untracked files and committing them? Are there other recommended ways to automate Git commits in a SnapLogic pipeline or related automation setup? Any guidance examples or best practices would be greatly appreciated. Thanks, Sneha
Solved
SnehaTB
New Contributor
16 days ago
31Views
0likes
1Comment
Pagination and nextCursor in header
Hello all, I'm using a HTTP Client snap to retrieved a few thousands of records, and I need to use pagination. The system that I'm calling is using cursor based pagination. If the number of elements returned is higher than the limit defined, the response header will contain a "nextCursor" value that I need to use as parameter to the "cursor" key for the next call, and so on until no more "nextCursor". This should be working fine, however I can't seem to get the content of the response header for my next call. When I use Postman I can see that there is a header returned, and the value that I need is stored under the key "X-Pagination-Next-Cursor" and not "nextCursor" as I expected. How can I access the values of the header? In the Snap itself, in the Pagination section, there is a "Override headers" part that I tried to configure by mapping the "cursor" key with either $nextCursor, $headers.nextCursor or $headers.X-Pagination-Next-Cursor, but nothing works, I'm only getting the records from the first page, there is no failure and no pagination. Thanks in advance for any help! JF
Solved
jfpelletier
Contributor
26 days ago
93Views
0likes
2Comments
How to get filename from file reader
I need to get the name of the file read by the file reader snap and use it as part of the data downstream. Really the goal to save the file name as part of the data pulled from a file. Screen snippet attached here. I have spent some time looking into this but there is no obvious method to me. Please I will appreciate any input and recommendations. Thanks.
Solved
omiaye
New Contributor II
2 months ago
78Views
0likes
2Comments
Generate expression file from database query
For some data transformations I would like to use an expression file that is generated each night, instead of querying a SQL database everytime the pipeline is started. I already have data available in the database and now I need to get the data transformed in the expression file JSON format, but I am stuck on getting the right ouput. Coming from a XML oriented environment (with extensive knowledge in XSL but not so much JSON) I have quite some issues with switching to snaps and JSON... Data sample (JSON) from the database [ { "code": "ARTICLEGROUP", "source": "JLG", "target": "10" }, { "code": "COMMODITYCODE", "source": "31251501", "target": "0" }, { "code": "COUNTRYCODE", "source": "AF", "target": "AF" }, { "code": "COUNTRYCODE", "source": "AL", "target": "AL" }, { "code": "COUNTRYCODE", "source": "DZ", "target": "DZ" }, { "code": "COUNTRYCODE", "source": "AS", "target": "AS" }, { "code": "COUNTRYCODE", "source": "AD", "target": "AD" }, { "code": "COUNTRY_ISOCODE", "source": "ARE", "target": "AE" }, { "code": "COUNTRY_ISOCODE", "source": "AFG", "target": "AF" }, { "code": "COUNTRY_ISOCODE", "source": "ALA", "target": "AX" }, { "code": "COUNTRY_ISOCODE", "source": "ALB", "target": "AL" }, { "code": "UOM", "source": "EA", "target": "pi" }, { "code": "UOM", "source": "M", "target": "me" }, { "code": "UOM", "source": "BG", "target": "za" } ] Desired output { "ARTICLEGROUP" : { "JLG": "10" }, "COMMODITYCODE" : { "31251501": "0" }, "COUNTRYCODE" : { "AF": "AF", "AL": "AL", "DZ": "DZ", "AS": "AS", "AD": "AD" }, "COUNTRY_ISOCODE" : { "ARE": "AE", "AFG": "AF", "ALA": "AX", "ALB": "AL" }, "UOM" : { "EA": "pi", "M": "me", "BG": "za" } , getValue : (type, source) => this[type][source] } Anyone can point me in the right direction? Have tried multiple things already, but I can't get the "arrays" right for some reason.
Solved
XanderVR
New Contributor
2 months ago
expression_library
json
86Views
0likes
3Comments
Can we generate XML file in pretty print format using native snapLogic snaps?
Hi Team, I was curious to know if anybody has worked on a use case where they are generating an XML file in pretty print format? We do have "pretty-print" option in JSON formatter however the same is not available in XML formatter snap. Any suggestions? Thanking in advance. Best Regards, Darsh
Solved
darshthakkar
Valued Contributor
2 months ago
707Views
0likes
3Comments