Forum Discussion

LVanceQH's avatar
LVanceQH
New Contributor
3 years ago

Trying multiple inputs into Directory Browser Snap

Hello Community!
I am trying to design a pipeline which takes a list of S3 URLs generated by a Mapper and uses the Directory Browser snap to produce output. I am seeing that the pipeline runs, but only the first URL in the list is picked up by the Directory Browser. Is there a way to cycle through the list of all the URLs?

2 Replies

  • darshthakkar's avatar
    darshthakkar
    Valued Contributor
    3 years ago

    Try using a MultiFile Reader instead of a directory browser and let me know if that works!

    A directory can ONLY have 1 output view so it will work when you’re hitting a location where all these files reside. Once you hit that location, you’ll have to used a router to route those different URLs.
    I would recommend going ahead with a Multi File Reader snap instead.

  • PratapKr_Gayen's avatar
    PratapKr_Gayen
    New Contributor II
    3 years ago

    You need to break down the pipeline in two segment.

    1. parent pipeline will read/generate list of URLs and use pipeline execute snap to call child pipeline.
    2. Child pipeline will contain the directory browser snap and you need to use pipeline parameter to pass the URL to child pipeline.

    hope it will solve the issue.