Not able to configure the HAS NEXT with HTTP Client snap

[terrible_towel]

Here's the response from my request:

There's nothing for me to work with, since only total_count is being returned. The $entity in the header doesn't return the properties I need to cycle through the pages (offset property, 0 by default).

I've tried the "snap.out.totalCount" but it seems I can't properly configure that. Can someone help me with setting up the next page?

I am able to do it in Postman, adding a "&offset=X" to the endpoint.

Thanks!

[ddellsperger]

I'm not sure what exceeds_total_count is in this case, but you might be able to leverage that, if not you likely will have to check the size of data $entity.data.length for whatever page size you're requesting.

If, for instance your page size is 100, you could put $entity.data.length >= 100 in your has next, and then your next url would have to be something like $url + "&offset=" + 100 * (snap.out.totalCount + 1) since the next page is calculated before the output page is processed, so it will be one off. If on the other hand, you know how many pages you need specifically, you can set the has next to simply true and then set the page limit in the snap settings.

[terrible_towel]

I do know the total count as it comes back from each request. I've tried setting the has next to 'true' and then the total pages to fetch = &entity.total_count, but it complains that 'entity' is undefined for some reason. What am I missing?

[ddellsperger]

your total pages to fetch will only be able to use input parameters, unfortunately not the data from the response. You won't be able to use the $entity within that, so you'll have to use the has next to determine if you've reached the end, you could do something like $entity.data.length >= 100 && (100 * (snap.out.totalCount + 1) <= $entity.total_count) in order to check for both 1) you've got all of the data expected in this response and 2) you haven't reached the limit of data.

[terrible_towel]

It seems like this approach does allow me to reach next page, but instead of fetching the rest of the dataset, it just re-fetches the same data from the 1st fetch. So it looks like the Has next expression is working, meaning it evaluates to true, but isn't able to pull the rest of the data. It seems like I'm failing at notifying the API that I want the 2nd batch onwards. So if I start with offset=0, then I'm not able to increment that value.