HTTP Client Snap - How to replace/override Raw entity value for next iteration (pagination)

[Vikas2022]

Hello,

In Snaplogic I am connecting to Elasticsearch API 7.17 via Apikey using HTTP Client Snap to read aggregate data

Below are the settings of the Snap

Request Method: GET
URI: https://elasticsearch-instance/_search
Entity Type: raw
Raw Entity:
{
< JSON Request Body>
}

Has Next:
$entity[‘aggregations’][‘by_customer’][‘after_key’][‘by_customer’] !=null

So as shown above, I am checking in Has Next property if a value is not null only then go to the next iteration. Next URI property will remain the same as URI. However, the catch is for the next iteration I want to replace/override the Raw entity property of the Snap with a different < JSON Request Body> which will include dynamic value coming from $entity[‘aggregations’][‘by_customer’][‘after_key’][‘by_customer’]

How can I achieve the above?

Reference link from Elastic API for the pagination

elastic.co

Composite aggregation | Elasticsearch Guide [7.17] | Elastic

@dmiller can you please help me

Thanks.

[dmiller]

Unfortunately, I haven’t used the HTTP Client Snap, so I don’t know if this is possible. I have to defer to engineering or someone else who has used it.

[Vikas2022]

Thanks @dmiller for the prompt reply.

If this requirement as per your knowledge can be achieved with any other SNAP like REST GET/POST please let me know so we can use that approach as well.

[ddellsperger]

None of the current Rest/Http Client snaps support paging via body entity, the typical way around that is to use a triggered task to deal with crafting the request and passing the parameters to that task via query parameters and notice to get to the next page via the output document from the triggered task.