Problem Accessing file(SFTP) using File Reader Snap

Hi All,

Am trying to create two pipelines with below information,

  1. Loading a csv file from sftp onto sql server - Created a pipeline containing below objects and problem is validation takes forever. Is this the correct snap or should I use ‘CSV Read/Write’ snap for it or suspect something is wrong with account settings?

SFTP Account created as Basic Auth,

User Name: ftpuser@52.86.106.54:22

File location path(sftp) which I verified is working when trying to access from browser,

  1. Loading a csv file from google drive onto sql server.

Tried using google spreadsheet reader snap and could only see excel files but not csv. Is this the correct snap or should I use REST snap pack for it.

Thanks in advance!

-Arun

In the 4.x code line, there isn’t a ‘CSV Read/Write’ Snap. File Read to CSV Parser is the correct configuration. Performance issues may require Support to investigate.

I don’t believe the Google Spreadsheet Reader reads CSV files, but I will double check with Development.

Thanks!!

Even I tried below as per the file reader snap documentation,

For google drive access,
Can we use REST snap pack for accessing csv files on google drive. Is this the right approach now??

Mapper Snap properties for above reply on the pipeline

To use $filename in the File field, you need to toggle on the expression button (=). Without it, it is treated as a string.

That worked but it takes me back to validation which runs forever,

Should I email customer support(support@snaplogic.com) to check for any performance issues.

Yes, that would be the correct action to take.

Hi - you need to remove the “CSV Formatter” Snap - the “CSV Parser” Snap converts the document stream to JSON, and the “SQL Server - insert” Snap expects JSON as an input. You can see that the “CSV Parser” Snap’s output is a diamond (binary), which won’t connect up to the “SQL Server - insert”'s circle (JSON).

If the CSV is in the exact structure of the SQL Server table (same field names) then you might be able to connect the “CSV Parser” directly to the “SQL Server - insert”. Otherwise you can put a Mapper in between to map between the CSV structure and SQL Server table structure.

Good luck!