Trying multiple inputs into Directory Browser Snap

[LVanceQH]

Hello Community!
I am trying to design a pipeline which takes a list of S3 URLs generated by a Mapper and uses the Directory Browser snap to produce output. I am seeing that the pipeline runs, but only the first URL in the list is picked up by the Directory Browser. Is there a way to cycle through the list of all the URLs?

[darshthakkar]

Try using a MultiFile Reader instead of a directory browser and let me know if that works!

A directory can ONLY have 1 output view so it will work when you’re hitting a location where all these files reside. Once you hit that location, you’ll have to used a router to route those different URLs.
I would recommend going ahead with a Multi File Reader snap instead.

[PratapKr_Gayen]

You need to break down the pipeline in two segment.

1. parent pipeline will read/generate list of URLs and use pipeline execute snap to call child pipeline.
2. Child pipeline will contain the directory browser snap and you need to use pipeline parameter to pass the URL to child pipeline.

hope it will solve the issue.