Pagination in REST API

Question

A very good explanation of how to implement pagination in REST API in this below blog post:

SnapLogic – 4 Feb 16

REST GET and the SnapLogic Public APIs for Pipeline Executions

As a part of a wider analytics project I’m working on, analyzing runtime information from the SnapLogic platform, I chose ...

When I try to implement the same(I am extracting data from survey monkey api), I have noticed that when the “Next URL” is executing it is ignoring the query parameters provided.As shown in below image, when service URL is executing it’s getting right records considering all input query parameters i.e total:3043 but when “Next URL” is hitting it is extracting all records i.e. total : 61896

How to include query parameters in “Next URL”  to get required data?

ptaylor · Answer

Please share your configuration of the Snap’s settings.

siva_venna · Answer

below is configuration of Snap’s settings.
service URL: https://api.surveymonkey.net/v3/collectors/xxxxx/responses/bulk

In the “Next URL” i have replaced “offset” with “page” api is throwing error as bad request, may be api don’t accept parameter as offset.error below

ptaylor · Answer

I can’t see the entire expression for Next URL in your screen capture, but it seems more complicated than it needs to be.  Try this: “https://api.surveymonkey.net/v3/collectors/xxxxx/responses/bulk?per_page=” + ($entity.per_page) + “&amp;page=” + ($entity.page + 1)

ptaylor · Answer

Actually, after looking at the Survey Monkey API docs, I think it’s even easier. Try this:
Has next:    ‘next’ in $entity.links
Next URL:   $entity.links.next

siva_venna · Answer

Thank you ptaylor, for timely responses.
I tried with specified Has next and Next URlL parameters but no luck

As I have stated earlier in the second iteration it’s fetching all the total records ignoring query parameters.

Forum Discussion

Pagination in REST API

9 Replies

Recent Discussions

Google Sheets Subscribe questions

Basic string transformations not working

Can we generate XML file in pretty print format using native snapLogic snaps?

Multipart Reader failure - 'content-type' was not found

Unable to write sldb file to file system