Forum Discussion

Pakhi's avatar
Pakhi
New Contributor III
3 years ago

Pagination in HTTP Client snap

Hi All, I am working with HTTP client snap and this is my first snap in the pipeline and i am using GET method to read data from API. I have to make sure data from all pages gets captured from so...
jhipp's avatar
jhipp
New Contributor
3 years ago

I truncated the results for simplicity to show there’s no total_results or pages in my response. I know there are 3 pages of data (~1300 results) because I’ve run API request through postman.

To answer your first question, the only way I know there’s more to request is if there are 500 results returned on a page (max defined by the API), then I’d try to pull the next page. Rinse and repeat until the results returned is not 500. I realize there’s a potential for a campaign to have a number of results divisible by 500 and will make a call that will result in 0 responses.

ptaylor's avatar
ptaylor
Employee
3 years ago

Thanks. I understand.

This isn’t as easy as it should be, but try this:

  • Check Enable debug
  • For page in the Override Parameters, set the expression to parseInt($_debug.request.requestLine.match("page=([0-9]+)")[1]) + 1
  • jhipp's avatar
    jhipp
    New Contributor
    3 years ago

    That worked like a charm. I’ll keep that in my back pocket for future (less than stellar) APIs 🙂