How to Configure Pagination and Specify Next URL in HTTP Client Snap for API Data Retrieval

Matthew H. · 2026-06-16T10:14:10.607Z

I'm having trouble implementing pagination in my HTTP Client Snap. I have 'Has next' in Pagination set to $.entity.meta.page.hasMore == true but it doesn't seem to be iterating. How do I specify the next URL to export all pages from the API?

Welcome to the new Integration Nation!

10 comments

· Sorted by Oldest

Marwen M.
·
·
Hi Matthew, If you only configure: $.entity.meta.page.hasMore == true. => in Has next, SnapLogic knows another page exists, but it may not know how to retrieve it. I'd recommand checking the API definition;
Option 1: API returns a next-page URL
{ "entity": { "meta": { "page": { "hasMore": true, "nextUrl": "https://api.example.com/items?page=2" } } } }
=> $.entity.meta.page.hasMore => $.entity.meta.page.nextUrl
Option 2: API uses page numbers
{ "entity": { "meta": { "page": { "hasMore": true, "pageNumber": 1 } } } }
=>$.entity.meta.page.hasMore => $.entity.meta.page.pageNumber + 1
Option 3: API uses offset/limit:
{ "hasMore": true, "offset": 100, "limit": 100 }
=> Has next: $.hasMore => Next offset: $.offset + $.limit
Matthew H.
·
·
So mine looks like the:

Matthew H.

page":
{
     "perPage": 40
     "from": "2326"
     "to": "2279"
     "hasMore": true
}

Matthew H.
·
·
Are you saying I replace the $.entity.meta.page.hasMore == true line in my snap configuration?
Duane v.
·
·
Hi Matthew H. Has Next — keep it exactly as is, it's correct:
$.entity.meta.page.hasMore == true
Next URL — need to construct the next URL using from or to as a cursor. The pattern is usually:
"https://their-api-base-url.com/their/endpoint?from=" + $.entity.meta.page.to
The to value (2279) typically becomes the starting point for the next page — the API is likely paginating backwards through record IDs. To answer your direct question: No, do NOT replace the Has Next line. Keep it. Just add the Next URL field with the constructed URL above. --- To confirm the exact cursor field, should check your API docs — specifically whether the next page call uses from=<to_value> or cursor=<to_value> or something else. But the $.entity.meta.page.to value is almost certainly the right cursor to use. "What does the base URL of your API call look like, and is there a from or cursor query parameter?" — that'll confirm the Next URL expression pattern. 🙂
Matthew H.
·
(edited)·
Thanks Duane v. and Scott O.. The problem is that I can't fin the 'Next URL' field in the HTTP Client snap. There's one in the 'Rest Get' Snap but the docs say that one is being deprecated.
Matthew H.
·
·
Is 'Next URL' there and I'm just not seeing it?
Spiro T.
·
(edited)·
Matthew H. if you check the Override URI, the Next URI will appear. So the URI will be overridden with the specified Next URI for each pagination request.
Matthew H.
·
·
Thank you Spiro T.! I've been going crazy trying to find that. That's been my experience with SnapLogic; spending hours trying to solve a problem only to find I just had to check one more box to solve everything!
🙇‍♂️1
🙌1
Alan L.
·
·
that doesn't sound too bad after all

Marwen M.
·
·
Hi Matthew, If you only configure: $.entity.meta.page.hasMore == true. => in Has next, SnapLogic knows another page exists, but it may not know how to retrieve it. I'd recommand checking the API definition;
Option 1: API returns a next-page URL
{ "entity": { "meta": { "page": { "hasMore": true, "nextUrl": "https://api.example.com/items?page=2" } } } }
=> $.entity.meta.page.hasMore => $.entity.meta.page.nextUrl
Option 2: API uses page numbers
{ "entity": { "meta": { "page": { "hasMore": true, "pageNumber": 1 } } } }
=>$.entity.meta.page.hasMore => $.entity.meta.page.pageNumber + 1
Option 3: API uses offset/limit:
{ "hasMore": true, "offset": 100, "limit": 100 }
=> Has next: $.hasMore => Next offset: $.offset + $.limit
Matthew H.
·
·
So mine looks like the:

Matthew H.

page":
{
     "perPage": 40
     "from": "2326"
     "to": "2279"
     "hasMore": true
}

Matthew H.
·
·
Are you saying I replace the $.entity.meta.page.hasMore == true line in my snap configuration?
Duane v.
·
·
Hi Matthew H. Has Next — keep it exactly as is, it's correct:
$.entity.meta.page.hasMore == true
Next URL — need to construct the next URL using from or to as a cursor. The pattern is usually:
"https://their-api-base-url.com/their/endpoint?from=" + $.entity.meta.page.to
The to value (2279) typically becomes the starting point for the next page — the API is likely paginating backwards through record IDs. To answer your direct question: No, do NOT replace the Has Next line. Keep it. Just add the Next URL field with the constructed URL above. --- To confirm the exact cursor field, should check your API docs — specifically whether the next page call uses from=<to_value> or cursor=<to_value> or something else. But the $.entity.meta.page.to value is almost certainly the right cursor to use. "What does the base URL of your API call look like, and is there a from or cursor query parameter?" — that'll confirm the Next URL expression pattern. 🙂
Matthew H.
·
(edited)·
Thanks Duane v. and Scott O.. The problem is that I can't fin the 'Next URL' field in the HTTP Client snap. There's one in the 'Rest Get' Snap but the docs say that one is being deprecated.
Matthew H.
·
·
Is 'Next URL' there and I'm just not seeing it?
Spiro T.
·
(edited)·
Matthew H. if you check the Override URI, the Next URI will appear. So the URI will be overridden with the specified Next URI for each pagination request.
Matthew H.
·
·
Thank you Spiro T.! I've been going crazy trying to find that. That's been my experience with SnapLogic; spending hours trying to solve a problem only to find I just had to check one more box to solve everything!
🙇‍♂️1
🙌1
Alan L.
·
·
that doesn't sound too bad after all