Pagination in REST API

#1

A very good explanation of how to implement pagination in REST API in this below blog post:
https://www.snaplogic.com/blog/rest-get-and-the-snaplogic-public-apis

When I try to implement the same(I am extracting data from survey monkey api), I have noticed that when the “Next URL” is executing it is ignoring the query parameters provided.As shown in below image, when service URL is executing it’s getting right records considering all input query parameters i.e total:3043 but when “Next URL” is hitting it is extracting all records i.e. total : 61896

capture2
capture2.JPG818×164 43.3 KB

How to include query parameters in “Next URL” to get required data?

#2

Please share your configuration of the Snap’s settings.

#3

below is configuration of Snap’s settings.
service URL: https://api.surveymonkey.net/v3/collectors/xxxxx/responses/bulk

pagination
pagination.JPG1191×590 70.3 KB

In the “Next URL” i have replaced “offset” with “page” api is throwing error as bad request, may be api don’t accept parameter as offset.error below

pagination
pagination.JPG1058×147 31.1 KB

#4

I can’t see the entire expression for Next URL in your screen capture, but it seems more complicated than it needs to be. Try this: “https://api.surveymonkey.net/v3/collectors/xxxxx/responses/bulk?per_page=” + ($entity.per_page) + “&page=” + ($entity.page + 1)

#5

Actually, after looking at the Survey Monkey API docs, I think it’s even easier. Try this:

Has next: ‘next’ in $entity.links
Next URL: $entity.links.next

#6

Thank you ptaylor, for timely responses.
I tried with specified Has next and Next URlL parameters but no luck
snapset

response_
response_.jpg1109×405 137 KB

As I have stated earlier in the second iteration it’s fetching all the total records ignoring query parameters.

#7

Unfortunately, the Survey Monkey API is omitting all of the other query parameters that are not paging-related from the URLs under ‘links’. So you’ll have to add them to the Next URL:

$entity.links.next + “&start_created_at=” + Date.now().minusDays(10) + “&end_created_at=” + Date.now().minusDays(1) + “&sort_by=date_modified&sort_order=DESC”

#8

I have tried this option earlier to posting in the community, it works when you specify Next URL with little modification as :
$entity.links.next + “&start_created_at=” + Date.now().minusDays(10) + “&end_created_at=” + Date.now().minusDays(1)

I suggest you to include this exception in the documentation for REST Snap i.e. Survey Monkey API will not accept “offset” value and it will omit query parameters.

Thank you ptaylor :+1:

#9

@Siva_Venna, a ticket has been created for the Doc team.