Forum Discussion

jfpelletier's avatar
jfpelletier
Contributor
3 months ago
Solved

Pagination and nextCursor in header

Hello all, I'm using a HTTP Client snap to retrieved a few thousands of records, and I need to use pagination. The system that I'm calling is using cursor based pagination. If the number of element...
  • Scott's avatar
    2 months ago

    Hi JF - 

    Based on the documentation and your specific use case with cursor-based pagination, here's how to properly access response headers in the HTTP Client snap pagination configuration:
    Accessing Response Headers in Pagination
    For cursor-based pagination where the next cursor is in a response header like "X-Pagination-Next-Cursor", you need to configure the pagination settings as follows:
    1. Has Next Configuration
    In the Has Next field, use an expression to check if the header exists:

    $headers['X-Pagination-Next-Cursor'] != null

    1. Next URI Configuration
      In theNext URIfield, construct your URL with the cursor parameter from the header:

    "https://your-api-endpoint.com/api/data?cursor=" + $headers['X-Pagination-Next-Cursor']

    Key Points for Header Access

    • Use $headers object: Response headers are accessible through the $headers object in pagination expressions
    • Exact header name: Use the exact header name as returned by the API (in your case: X-Pagination-Next-Cursor)
    • Case sensitivity: Header names are case-sensitive, so ensure you match exactly what the API returns

    Complete Example Configuration
    For your specific scenario:
    Has Next:

    $headers['X-Pagination-Next-Cursor'] != null && $headers['X-Pagination-Next-Cursor'] != ""

    Next URI:

    "https://your-base-url/endpoint?limit=1000&cursor=" + $headers['X-Pagination-Next-Cursor']

    Alternative Approaches
    If the above doesn't work, you can also try:

    • $response.headers['X-Pagination-Next-Cursor']
    • Enable Debug mode in the HTTP Client snap to see all available response data structure

    Override Headers Section
    The "Override headers" section you mentioned is typically used to modify request headers for subsequent pagination calls, not to access response headers. For accessing response header values in pagination logic, use the $headers object in the Has Next and Next URI fields as shown above.
    This approach should allow you to properly implement cursor-based pagination using the header values returned by your API. I used SnapGPT to support formulating this response for you.