Downloading a base64 zip file using SOAP

Hi All,

I am trying to use SOAP call to download a zip file. However, I cannot download and get the file I need correctly. Here is a screenshot of my pipeline.
Capture

I took out the $content, which contains a string that is (at least looks like) base64 encoded data. The “Content-Type” is null and “Content-Transfer-Encoding” is
“binary”.

Then no matter how I decode this string, whether by using Base64.decode() or Base64.decodeAsBinary(), I cannot get the zip file I need. If I use Base64.decode($content), I get something like

I can see my file name within a bunch of words that I cannot read. And I think I should not use Base64.decodeAsBinary().

Could someone please tell me what is the correct way to download this zip?

Thanks in advance!

Hi @tonyzero,

I think the problem is in your ZipFile Writer. If I’m correct, you are receiving the file as a zip archive, using ZipFile Write will add that zip in another zip, unless of course, you are using a plane File Writer and you changed the label to ‘ZipFile Write’.

I would suggest that you take the content of the SOAP call with a Mapper and changed output as a binary. Also add a ‘content-type’ header with a value of ‘application/zip’, and use a File Writer (not ZipFile Writer) at the end.

Hope this helps,
Bojan

Hi @bojanvelevski

Thank you so much for your reply. I followed your instruction but still did not get the results I needed. I am kind new to this so please bear with me if I have some “silly” questions.

So the WSDL service that I am using says “Downloads the ESS job output and the logs as a zip file.” So I am assuming that whatever decoded from the “$content” should be a zip file.

And for the mapper, this is what I have for now. Not quite sure if this is correct.

For File Writer, I tried test.zip or test.txt but neither works.

If I output this file as zip, the I got the following error.
Central Directory

If I output this as test.txt, then I got all these unknown characters with my file name in it.

I am really confused on how to process with this procedure. Could you please guide me on that again.

So much appreciated!

Unfortunately, when an output document object has a field whose value is binary data (a byte array), the output preview will display it incorrectly as “Base64 encoded data”, when it’s not actually encoded at all. It’s just the data. That might be what’s happening here. The best way to diagnose this would be to show us a screenshot of the Mapper that immediately follows the SOAP Execute. Please expand the Input Schema and the Input Preview so that the content field in question is visible. Let’s start with that.

Hi @ptaylor ,

Thank you so much for your reply. This is what I got for my pipeline.

For the first SOAP snap, this is the preview results.I believe the “content” part is what I am looking for to decode.

Then just as what shown above, I connect a Filter to soap. Here is the setting for filter.
filter

Then I connect it with a mapper to decode it following Bojan’s suggestion.

Then the preview results still looks weird to me.

Then I connect it with a File Writer (but not sure what format I should write to).

This is what I got so far. I know there must be something wrong with it but could not figure it out.

Thank you so much for helping me on this.

Hi @tonyzero,

I don’t think you should decode the content like that, as far as I understood, you are getting the .zip file from the SOAP endpoint, which returns binary data, then what you should really do is just map the $content of the SOAP as $content, you shouldn’t decode anything in the mapper. Also, your $['content_type'] target path is not valid, you should write $['content-type'] not $['content_type'] with an underscore.

I’m a bit confused by the SOAP Execute response. It’s reporting the Content-Transfer-Encoding as “binary”, which essentially means “not encoded”. The actual content “UEsDBBQA…” looks like it actually is base64-encoded. If so, try this:

In your Mapper, change the expression to Base64.decodeAsBinary($content). Then in the Views tab of the Mapper settings, modify the output view’s type from Document to Binary. Attach a File Writer to write the content to a .zip file.

Hi @j.angelevski ,

Thank you so much for your reply. It actually works with Base64.decodeAsBinary($content) and I got what I need.

Hi @ptaylor ,

I followed you suggestion and it really works. I used Base64.decodeAsBinary($content) together with application/zip → $[‘content-type’] and got the txt file in a zip.
However, I am still not sure why I should use Base64.decodeAsBinary() to decode something what really looks like a base64 encoded string.

Thank you very much!

Great!

Base64.decodeAsBinary() is the fundamental function: converting a base64-encoded String to a byte array containing the binary data (in this case, the zip compressed data).

Baes64.decode() is actually a compound function: it starts by calling decodeAsBinary to convert the base64 String to a byte array, then converts that byte array to a String using the UTF-8 charset. In this case, the data in the byte array isn’t a UTF-8-encoded string, so that second conversion is just wrong and doesn’t result in anything meaningful. You need to first decompress the data in the byte array before converting it back to text.

Hope that helps.

2 Likes

Hi @ptaylor ,

Thank you so much. That actually helps a lot!