cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

What is the best way to loop through data from a file... like an csv?

swright
New Contributor III

I have a csv file with person ID numbers in one column. I need to use the id number to act on person records in the cloud via REST.

What is the best way to setup a pipeline to get an ID, do some processing, then execute a REST snap, then go back and do the same for the next ID and continue looping through the IDs?

Thanks,
Scott

1 ACCEPTED SOLUTION

sg_sripathi
New Contributor III

@swright, if you refer to below pipeline. It will make two REST API call to two different end point (URL constructed using input) and with HTTP body content as you see in the example. So basically the REST snap will take care of processing each document for you.

image

View solution in original post

11 REPLIES 11

pkk_1
New Contributor

Hi Scott,

You can separate out the logic and processing in different pipelines.
1] Pipeline 1 : Master/Parent pipeline that reads the csv file and extracts the records and pass ID as pipeline parameter to child Pipeline 2. If there are huge records, you can enable re use and configure the pool size as required.
2] Pipeline 2: To act on the ID received in the pipeline parameter and execute REST snap

Thank you

swright
New Contributor III

Hi Pooja,

I tried that with File Reader > CSV Parser > Sequence > Mapper > Pipeline Execute.

It didnโ€™t pass the parameter to the child pipeline. Is there something special that I need to do to send the parameter to the child?

Thanks,

Scott

pkk_1
New Contributor

Hi Scott,

In Pipeline executeโ€™s Parameter Name you need to set the ID(can be any name) and in the Parameter Value enable the expression and access the ID that you want to read.

Do share the sample csv file.

Thanks,
Pooja

swright
New Contributor III

Hi Pooja,

Do I somehow use the sequence number from the sequence snap as an index for a json array generated from the csv?

Thanks,

Scott

(Attachment sl_test.csv is missing)