Forum Discussion
acmohan023
5 years agoNew Contributor
@rsramkoski - Have attached the sample input structure and output structure for your reference. (I had pasted request and response structure and it is not visible).
InputAndOutpultSample.txt (1.3 KB)
As file Size is huge , tried to split the flow into multiple pipelines (to process it in chunk and release memory) :
- Read the file from SFTP and XML Parse to split the data . Group by N and call 2nd pipeline
- split group , by sort by field ( input field in attached file) call 3rd pipeline
- write files smaller chunk to local sftp
- On completion of all above 3 pipeline , read small chunk files based on file name and write consolidated file per group by field (input field in attached file)
Please suggest if there is any other approach to resolve this issue.
acmohan023
5 years agoNew Contributor
Team,
Can you please help and let me know if any details required.
- acmohan0235 years agoNew Contributor
can you please guide to resolve this issue.