What is the best way to loop through data from a file... like an csv?

I have a csv file with person ID numbers in one column. I need to use the id number to act on person records in the cloud via REST.

What is the best way to setup a pipeline to get an ID, do some processing, then execute a REST snap, then go back and do the same for the next ID and continue looping through the IDs?

Thanks,
Scott

Hi Scott,

You can separate out the logic and processing in different pipelines.
1] Pipeline 1 : Master/Parent pipeline that reads the csv file and extracts the records and pass ID as pipeline parameter to child Pipeline 2. If there are huge records, you can enable re use and configure the pool size as required.
2] Pipeline 2: To act on the ID received in the pipeline parameter and execute REST snap

Thank you

Hi Pooja,

I tried that with File Reader > CSV Parser > Sequence > Mapper > Pipeline Execute.

It didn’t pass the parameter to the child pipeline. Is there something special that I need to do to send the parameter to the child?

Thanks,

Scott

Hi Scott,

In Pipeline execute’s Parameter Name you need to set the ID(can be any name) and in the Parameter Value enable the expression and access the ID that you want to read.

Do share the sample csv file.

Thanks,
Pooja

Hi Pooja,

Do I somehow use the sequence number from the sequence snap as an index for a json array generated from the csv?

Thanks,

Scott

(Attachment sl_test.csv is missing)

sl_test.txt (117 Bytes)

Here is some sample csv data as requested.

Thanks,
Scott

@swright, not sure why you would require a complex setup. Is the processing complicated to be had in a separate pipeline. Most of the snaps process at document level, so you don’t have to loop, including REST snaps.

Hi Sripathi,

I’m new to Snaplogic. I would love to have a simple setup. The CSV parser snap produces a json file with a separate array element for each id. I need to create endpoints for each ID to and pass the endpoints to REST snaps. It seems like I need to execute the REST snap once for each url that is created using the PersonNumber from the csv. In many programing languages this would be done using a loop. It would take the first PersonNumber, get/create the endpoints for the REST snaps and run the REST snaps then move on to the next PersonNumber and do the same with each PersonNumber until there are no more PersonNumbers in the array.

I don’t know how to do this. Are there any examples?

Thanks,

Scott

@swright, if you refer to below pipeline. It will make two REST API call to two different end point (URL constructed using input) and with HTTP body content as you see in the example. So basically the REST snap will take care of processing each document for you.

Thanks Sripathi! That worked perfectly!

Scott

@sg_sripathi , I’m curious what your csv parser setup looks like. Would you mind posting that, or the pipeline export please?

@acesario it’s as simple as dragging ang dropping the csv parser snap on this example as the data is comma separated which is by default set on snap.

If there are any other separators or if may want to include or remove header then below is the docbase link for more information on the snap configuration.
https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1438538/CSV+Parser

1 Like