How to Configure Pagination and Specify Next URL in HTTP Client Snap for API Data Retrieval

Matthew H. · 2026-06-16T10:14:10.607Z

I'm having trouble implementing pagination in my HTTP Client Snap. I have 'Has next' in Pagination set to $.entity.meta.page.hasMore == true but it doesn't seem to be iterating. How do I specify the next URL to export all pages from the API?

Welcome to the new Integration Nation!

How to Configure Pagination and Specify Next URL in HTTP Client Snap for API Data Retrieval | Integration Nation

10 comments

· Sorted by Oldest

Marwen M.
·
·
Hi Matthew, If you only configure: $.entity.meta.page.hasMore == true. => in Has next, SnapLogic knows another page exists, but it may not know how to retrieve it. I'd recommand checking the API definition;
Option 1: API returns a next-page URL
{ "entity": { "meta": { "page": { "hasMore": true, "nextUrl": "https://api.example.com/items?page=2" } } } }
=> $.entity.meta.page.hasMore => $.entity.meta.page.nextUrl
Option 2: API uses page numbers
{ "entity": { "meta": { "page": { "hasMore": true, "pageNumber": 1 } } } }
=>$.entity.meta.page.hasMore => $.entity.meta.page.pageNumber + 1
Option 3: API uses offset/limit:
{ "hasMore": true, "offset": 100, "limit": 100 }
=> Has next: $.hasMore => Next offset: $.offset + $.limit
Matthew H.
·
·
So mine looks like the:

Matthew H.

page":
{
     "perPage": 40
     "from": "2326"
     "to": "2279"
     "hasMore": true
}

Matthew H.
·
·
Are you saying I replace the $.entity.meta.page.hasMore == true line in my snap configuration?
Duane v.
·
·
Hi Matthew H. Has Next — keep it exactly as is, it's correct:
$.entity.meta.page.hasMore == true
Next URL — need to construct the next URL using from or to as a cursor. The pattern is usually:
"https://their-api-base-url.com/their/endpoint?from=" + $.entity.meta.page.to
The to value (2279) typically becomes the starting point for the next page — the API is likely paginating backwards through record IDs. To answer your direct question: No, do NOT replace the Has Next line. Keep it. Just add the Next URL field with the constructed URL above. --- To confirm the exact cursor field, should check your API docs — specifically whether the next page call uses from=<to_value> or cursor=<to_value> or something else. But the $.entity.meta.page.to value is almost certainly the right cursor to use. "What does the base URL of your API call look like, and is there a from or cursor query parameter?" — that'll confirm the Next URL expression pattern. 🙂
Matthew H.
·
(edited)·
Thanks Duane v. and Scott O.. The problem is that I can't fin the 'Next URL' field in the HTTP Client snap. There's one in the 'Rest Get' Snap but the docs say that one is being deprecated.
Matthew H.
·
·
Is 'Next URL' there and I'm just not seeing it?
Spiro T.
·
(edited)·
Matthew H. if you check the Override URI, the Next URI will appear. So the URI will be overridden with the specified Next URI for each pagination request.
Matthew H.
·
·
Thank you Spiro T.! I've been going crazy trying to find that. That's been my experience with SnapLogic; spending hours trying to solve a problem only to find I just had to check one more box to solve everything!
🙇‍♂️1
🙌2
Alan L.
·
·
that doesn't sound too bad after all

Marwen M.
·
·
Hi Matthew, If you only configure: $.entity.meta.page.hasMore == true. => in Has next, SnapLogic knows another page exists, but it may not know how to retrieve it. I'd recommand checking the API definition;
Option 1: API returns a next-page URL
{ "entity": { "meta": { "page": { "hasMore": true, "nextUrl": "https://api.example.com/items?page=2" } } } }
=> $.entity.meta.page.hasMore => $.entity.meta.page.nextUrl
Option 2: API uses page numbers
{ "entity": { "meta": { "page": { "hasMore": true, "pageNumber": 1 } } } }
=>$.entity.meta.page.hasMore => $.entity.meta.page.pageNumber + 1
Option 3: API uses offset/limit:
{ "hasMore": true, "offset": 100, "limit": 100 }
=> Has next: $.hasMore => Next offset: $.offset + $.limit
Matthew H.
·
·
So mine looks like the:

Matthew H.

page":
{
     "perPage": 40
     "from": "2326"
     "to": "2279"
     "hasMore": true
}

Matthew H.
·
·
Are you saying I replace the $.entity.meta.page.hasMore == true line in my snap configuration?
Duane v.
·
·
Hi Matthew H. Has Next — keep it exactly as is, it's correct:
$.entity.meta.page.hasMore == true
Next URL — need to construct the next URL using from or to as a cursor. The pattern is usually:
"https://their-api-base-url.com/their/endpoint?from=" + $.entity.meta.page.to
The to value (2279) typically becomes the starting point for the next page — the API is likely paginating backwards through record IDs. To answer your direct question: No, do NOT replace the Has Next line. Keep it. Just add the Next URL field with the constructed URL above. --- To confirm the exact cursor field, should check your API docs — specifically whether the next page call uses from=<to_value> or cursor=<to_value> or something else. But the $.entity.meta.page.to value is almost certainly the right cursor to use. "What does the base URL of your API call look like, and is there a from or cursor query parameter?" — that'll confirm the Next URL expression pattern. 🙂
Matthew H.
·
(edited)·
Thanks Duane v. and Scott O.. The problem is that I can't fin the 'Next URL' field in the HTTP Client snap. There's one in the 'Rest Get' Snap but the docs say that one is being deprecated.
Matthew H.
·
·
Is 'Next URL' there and I'm just not seeing it?
Spiro T.
·
(edited)·
Matthew H. if you check the Override URI, the Next URI will appear. So the URI will be overridden with the specified Next URI for each pagination request.
Matthew H.
·
·
Thank you Spiro T.! I've been going crazy trying to find that. That's been my experience with SnapLogic; spending hours trying to solve a problem only to find I just had to check one more box to solve everything!
🙇‍♂️1
🙌2
Alan L.
·
·
that doesn't sound too bad after all