Zip Snap file structure

[SuryaReddy]

I have a pipeline which reads data from an S3 location and writes to an SMB location with no transformations. The pipeline involves 2 main snaps, S3 reader connected to Zip write snap. The Zip write snap recreates the whole structure of the S3 location in the Zip file, which I would like to avoid and place all the files in either a root directory or a directory of my choice instead of copying the S3 structure.
As there is no transformation, I do not want to parse the files and parse back again. Is there a way of doing this?

[bojanvelevski]

Hi @SuryaReddy,

I cannot understand the reason why you have to write Zip file , and read it after? Why not a plain S3 read, and write to SMB? Are you using the same path from the S3, to set the file path to SMB? I’m just guessing. Example:

s3:///mybucket@s3.eu-test-1.amazonaws.com/test_files/csv/input.csv => smb://smb.Snaplogic.com:445/test_files/csv/input.csv

If this is the case, you can easily take the filename only, and define the target path on your own, if not, please send some more details.

[SuryaReddy]

Hi @bojanvelevski ,
The pipeline does this:
Read files (and there are multiple files to read) from S3 bucket ==> Write to an SMB Location as a zipped file.
When doing this, the zip file has the same folder structure as the S3, which is very deep. I am looking to avoid this deep structure and make a much more flat structure.

Example:
S3 file folder structure: Bucket → Folder1 → Folder2 → Folder3 → File
Zip file structure: Zip file name (User defined) → Bucket → Folder1 → Folder2 → Folder3 → File
Required: Zip file name (User defined) → Bucket → Folder1 → File

[bojanvelevski]

That’s because you are passing the content location as a target filepath. Use the following expression to get the filename only:

$['content-location'].split('/')[$['content-location'].split('/').length -1]

than, you can define the path as you like it to be. Example:

"smb://smb.Snaplogic.com:445/Bucket/Folder1/" + $['content-location'].split('/')[$['content-location'].split('/').length -1]

Don’t forget to enable the expressions functionality.

[SuryaReddy]

Test pipeline 0_2021_10_01.slp (3.0 KB)
I have uploaded a stripped down version of the pipeline, can you advise how to change the path you mention above?