This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
FAQ
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Running Rest API in a loop where Page count is fixed variable

Go to solution
adityamohanty
New Contributor II

‎02-03-2024 10:15 PM

Step -1 

Rest API to get the collection info looks like 

https://stage-data.xyz.com/v1/xyz-api/collections/models/info?page_size=100

Output:

{
    "total_count": 5099,
    "page_count": 51,
    "page_size": 100
}
Step -2
Rest API to get the model looks like
 
Output:
It will give model information of 1st page with 100 records.
 
Question - I want to run it 51 times (based on page count) in a loop to load entire records 
 
Solution:
 
I captured the page count & total count through mapper and my Get Rest Snap looks like 
HTTP Header : As per Need
Has next : snap.in.totalCount == $page_count
Next URL: "https://stage-data.xyz.com/v1/xyz-api/collections/models/" + (snap.in.totalCount + 1) + "?page_size=100" + '?skip=' + (snap.in.totalCount + 1).toString()
 
I am able to load only 100 records to my target. Please advise
0 Kudos
1 ACCEPTED SOLUTION

Go to solution
AleksandarAngel
Contributor III
In response to adityamohanty

‎02-06-2024 02:14 AM

Hello again,

As I mentioned above, this is not how a typical pagination would look like using the REST GET / HTTP Client Snap, since I'm not aware about any information in the response that can be used to set the Has Next and Next URL properties. 

And since that you have the total number of pages in the info request, this approach will generate as many documents (requests) as you have pages.

If your parameters are static you can map them in the Map - Pages Mapper Snap and pass them in the JSON Splitter in the Include Paths section so you can use them in the REST GET Snap.

Please refer to the attached pipeline and let me know if this helps you.

Regards,

Aleksandar.

View solution in original post

--sl-comm-pagination_2024_02_06.zip
5 REPLIES 5

Go to solution
AleksandarAngel
Contributor III
In response to adityamohanty

‎02-12-2024 12:43 AM

Yes, that is correct! The documents are being processed sequentially, one by one.

0 Kudos