Forum Discussion

ddangler's avatar
ddangler
New Contributor III
9 years ago

How to invoke child pipeline for 'N' times where 'N' is known only at run time in the parent pipeline?

Hello Forum,

I have a input document like this:

[
	{
		"count" : 10
	}

]

and I need to invoke child pipeline using Pipeline Execute snap 10 times. If count is 5, I need to execute child pipeline 5 times.

How can I achieve this? Thanks in Advance!

6 Replies

  • tstack's avatar
    tstack
    Former Employee

    Leveraging the REST Snap’s pagination functionality is definitely the way to go for your use case, but I wanted to make sure your original question was answered as well. The Pipeline Execute snap will kick off a child pipeline for every document it receives (if reuse is turned off). So, to invoke something N times, you’ll need to generate N documents. Generating documents can be achieved with the Sequence snap, if the number is coming from a pipeline parameter. If the number is coming from a document, you can use the ‘sl.range()’ method to create an array that is then split with a JSONSplitter to create the documents for the PipeExec snap.

    • ddangler's avatar
      ddangler
      New Contributor III

      @nganapathiraju,

      We are querying tickets from Zendesk API and it does not return all the records in one call but gives the url to use in the next HTTP call to get the next set of records.

      Sample response from the first call:

      {
        "tickets": [ ... ],
        "count": 1234,
        "next_page": "https://account.zendesk.com/api/v2/tickets.json?page=2",
        "previous_page": null
      }
      

      In this example, I need to make calls to their API until next_page is null or 12 times (1234 /100, because each call returns only max of 100 records).

      Here is the url to their documentation if you want more details: https://developer.zendesk.com/rest_api/docs/core/introduction#pagination

      Please throw some inputs on how this can be done? Thanks in Advance!