Forum Discussion

Vikas2022's avatar
Vikas2022
New Contributor II
3 years ago

HTTP Client Snap - How to replace/override Raw entity value for next iteration (pagination)

Hello,

In Snaplogic I am connecting to Elasticsearch API 7.17 via Apikey using HTTP Client Snap to read aggregate data

Below are the settings of the Snap

Request Method: GET
URI: https://elasticsearch-instance/_search
Entity Type: raw
Raw Entity:
{
< JSON Request Body>
}

Has Next:
$entity[‘aggregations’][‘by_customer’][‘after_key’][‘by_customer’] !=null

So as shown above, I am checking in Has Next property if a value is not null only then go to the next iteration. Next URI property will remain the same as URI. However, the catch is for the next iteration I want to replace/override the Raw entity property of the Snap with a different < JSON Request Body> which will include dynamic value coming from $entity[‘aggregations’][‘by_customer’][‘after_key’][‘by_customer’]

How can I achieve the above?

Reference link from Elastic API for the pagination

@dmiller can you please help me

Thanks.

3 Replies

  • dmiller's avatar
    dmiller
    Former Employee
    3 years ago

    Unfortunately, I haven’t used the HTTP Client Snap, so I don’t know if this is possible. I have to defer to engineering or someone else who has used it.

    • Vikas2022's avatar
      Vikas2022
      New Contributor II
      3 years ago

      Thanks @dmiller for the prompt reply.

      If this requirement as per your knowledge can be achieved with any other SNAP like REST GET/POST please let me know so we can use that approach as well.

  • ddellsperger's avatar
    ddellsperger
    Admin
    3 years ago

    None of the current Rest/Http Client snaps support paging via body entity, the typical way around that is to use a triggered task to deal with crafting the request and passing the parameters to that task via query parameters and notice to get to the next page via the output document from the triggered task.