cancel
Showing results for 
Search instead for 
Did you mean: 

Mulitple Files trying to be processed

FalleurJ
New Contributor

I have 2 files in a directory that need to be ran through my pipeline, how do i get it to find the next one in directory and run through the same pipeline as well and get the conversions done and for it to post where i need it to on my File Writers.

I beleive it is my TXT Parser, I need it to be able to parse multiple files. or if i can combine the files at the beginning.

image

image

File Poller picking up 2
image

File Reader reading 2
image

3 REPLIES 3

aleung
Contributor III

Which part of your pipeline isn’t working? Have you tried Multi File Reader?

Minovski
New Contributor II

If I understand the issue correctly, you need to parse and transform the files one by one.
You can use Multi File Reader snap to read those 2 files, you will need to add binary to document snap to encode the files. Use Pipeline Execute to send the files one by one. Here is an example:

image

Inside the Child pipeline that you will create you can parse your files and do your transformation.

sshaik
New Contributor III

Could you provide a sample Pipeline regarding to parse and transform the files one by one