09-29-2021 03:30 AM
I have a pipeline which reads data from an S3 location and writes to an SMB location with no transformations. The pipeline involves 2 main snaps, S3 reader connected to Zip write snap. The Zip write snap recreates the whole structure of the S3 location in the Zip file, which I would like to avoid and place all the files in either a root directory or a directory of my choice instead of copying the S3 structure.
As there is no transformation, I do not want to parse the files and parse back again. Is there a way of doing this?
10-01-2021 11:14 AM
Your current version of the pipeline, will write the zip file on SLDB under the name “TESTlOG.zip” . What am I suggesting is to use the expression above to get the actual filename you’re reading from S3, and use that filename to write the file on SMB. Example: