Rest GET - Next URL with page=# syntax with no metadata

Designing Pipelines
,
#1

Hello,

I’m calling a REST API which requires me to increment a ‘page’ value in order to get all the results. The issue is the API doesn’t return any metadata. For example:

1st call is: https:domain.com/object/page/1

Results:
{
entity:[
{…},{…},…
]
}

Has next: typeof $entity == ‘array’ && $entity.length > 0
Next URL: https:domain.com/object/page/2

The problem is how to I compute the 2 (then 3,4,5… etc) Is it possible to increment a value by 1 on every subsequent call? Or is it possible to access a variable that counts how many times the call has been made?

Thanks!

#2

For Next URL I believe you can use the expression:

'https:domain.com/object/page/' + (snap.out.totalCount + 1).toString() to have second request add 2, then 3, 4, 5, and so on.

1 Like
#3

Thanks @cjhoward18, that worked for me!

#4

@dmiller : cjhoward18’s response above, i.e.

For Next URL I believe you can use the expression:
'https:domain.com/object/page/' + (snap.out.totalCount + 1).toString() to have second request add 2, then 3, 4, 5, and so on.

should be added to the pagination portion of the REST Get documentation. I’m working with an API that also does not include the current page number, nor next offset in the response, so implementing pagination requires incrementing the offset. I had to spend 15 minutes searching community before I came across this tip.

#5

I’ll let the doc team know.