- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
‎02-03-2024 10:15 PM
Step -1
Rest API to get the collection info looks like
https://stage-data.xyz.com/v1/xyz-api/collections/models/info?page_size=100
Output:
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
‎02-06-2024 02:14 AM
Hello again,
As I mentioned above, this is not how a typical pagination would look like using the REST GET / HTTP Client Snap, since I'm not aware about any information in the response that can be used to set the Has Next and Next URL properties.
And since that you have the total number of pages in the info request, this approach will generate as many documents (requests) as you have pages.
If your parameters are static you can map them in the Map - Pages Mapper Snap and pass them in the JSON Splitter in the Include Paths section so you can use them in the REST GET Snap.
Please refer to the attached pipeline and let me know if this helps you.
Regards,
Aleksandar.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
‎02-05-2024 01:11 AM - edited ‎02-05-2024 01:27 AM
Hello @adityamohanty,
You could with the following approach:
- Generate a document per page. (Dynamically based on the page count and page size from the initial response)
sl.range(1,$page_count+1,1).map(x => {"page":x, "page_size":$page_size})
- Split the array using JSON Splitter Snap.
- REST GET request per each document.
I chose this approach since I'm not aware of the format of the response of the models endpoint. If there are pagination information, you could achieve the same using REST GET Snap or HTTP Client Snaps with proper pagination enabled.
You can refer to the pipeline attached below and let me know if this helps you.
Regards,
Aleksandar.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
‎02-05-2024 12:33 PM
Hello @Aleksandar_A
Thank you so much for the reply. I understood your approach but not sure how does it help to run it in a loop because following "Has Next" and "next url" is blank in REST GET request in your pipeline (Screenshot attached)
And I have few parameters and token which has been used to call the API and I can successfully call it out through mapper and used in REST GET snap. But when passing JSON Splitter as per the suggestion between Mapper and REST GET to split the array of pages, it's not capturing my parameters and token value to the REST GET Input. The only output of JSON Splitter, I can see is Page & Page Size. Any advise ?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
‎02-06-2024 02:14 AM
Hello again,
As I mentioned above, this is not how a typical pagination would look like using the REST GET / HTTP Client Snap, since I'm not aware about any information in the response that can be used to set the Has Next and Next URL properties.
And since that you have the total number of pages in the info request, this approach will generate as many documents (requests) as you have pages.
If your parameters are static you can map them in the Map - Pages Mapper Snap and pass them in the JSON Splitter in the Include Paths section so you can use them in the REST GET Snap.
Please refer to the attached pipeline and let me know if this helps you.
Regards,
Aleksandar.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
‎02-09-2024 11:08 AM
Thank You @Aleksandar_A. It worked for me. I just wanted to check one more thing. When we create no of documents to hit the API that no of times. Does it run sequentially ?
I mean 1 document hits the API and once it gets the data then 2 document hits the API ?
