This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
FAQ
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results forย 
Search instead forย 
Did you mean:ย 

REST API returning tar.gz

ryan
New Contributor

โ€Ž01-31-2022 08:56 AM

I am attempting to make an API call that returns a tar.gz file. I want to convert it to a csv and then upload it to an S3 bucket.
Note: S3 and aws is not set up yet but would like to be able to read it in and get that aspect down before they our team has our aws accounts applied.

Currently I have tried a few different methods but am not getting the expected results.

Rest Get >>> Mapper (entity/what I would think is the content data)>>>CSV Formatter>>>File Writer

I am obviously doing something wrong so any suggestions would be appreciated.

0 Kudos
1 REPLY 1

koryknick
Employee
Employee

โ€Ž01-31-2022 10:08 AM

Hi @ryan - welcome to the Community! Great first question! SnapLogic does not have a snap to read the TAR file, so you will need to use a Script snap to handle that portion of this request.

I came across this a while back and have a simple pipeline example using a Script snap to explode out the TAR data. This solution works fine for smaller TAR files but since it does everything in memory, it may consume large amounts of your node memory if the input file is large. This is attached here:
Read tar.gz file_2022_01_31.slp (12.3 KB)

One of my co-workers created a slightly different design that pushes the TAR file to the snaplex local node file system and uses Python to explode out the files within the file system then reads the contents back in. Hopefully this link works for you: Box

Read tar.gz file_2022_01_31.slp