Forum Discussion

Barbara's avatar
Barbara
New Contributor II
3 years ago

Recommended Approach for Multiple REST Get Snaps

I need to make multiple GET calls to the same API, aggregate and flatten the JSON record, and write one single csv file. Each GET snap execution parameters are the same except for the RecordType, which has six (6) different values.

The solution in video “Snaplogic Best Practices: Sub Pipelines and Guaranteed Delivery” at https://www.youtube.com/watch?v=RqjKkG4V354 may be part of the solution.

A ForEach pipeline with the list of RecordType values in JSON that is parsed and passed to the subpipeline with the GET snap using ForEach would work. How can the multiple responses from the GET snap pipeline be aggregated so they can be flattened and output into a single csv file?

pipeline

3 Replies

  • koryknick's avatar
    koryknick
    Employee
    3 years ago

    ForEach is a deprecated snap and we recommend that you use Pipeline Execute instead, which you could simply split the RecordType values and call the child pipeline for each of the values… you could even set the Pool Size as more than 1 to get some concurrency of the call.

    As for grouping the results, you could use Group By Fields, Group By N, or Gate snap, depending on how this pipeline is executed and the expected records into and out of the child pipeline.

  • Barbara's avatar
    Barbara
    New Contributor II
    3 years ago

    Thank you for the response. I am new to SnapLogic. Is there an example of a solution available that is close to the one that needed?

  • koryknick's avatar
    koryknick
    Employee
    3 years ago

    Here is a small example calling an API and aggregating the results into a single record.
    Community 13390 - Combine Multi-REST Results_2022_09_15.slp (9.1 KB)

    Note that I did not use a child pipeline or Pipeline Execute in this example - I’m simply calling the API multiple times. Then I’m using an Aggregate snap to pivot the results from multiple records to a single record with multiple columns.

    Hope this helps!