amarb
New Contributor II
Joined 8 years ago
User Profile
User Widgets
Contributions
Re: Connect SFTP server to fetch multiple files
Hi tstack, I’ve tried and able to succeded below three snaps to fetch files from SFTP server Directory Browser File Poller Multi File Reader Based on suggestions tried child pipeline to get filename and file content. I tried using underscore in the File Reader but still getting same error. My goal is to extract filename to find out region from the name and extract file content to map SQL table and load into SQL server. Appreciate your help!! Thanks,857Views0likes1CommentRe: Connect SFTP server to fetch multiple files
Thank you Robin and Del. I’ve tried both ways and still getting an error. Attached screenshots for your ref. Is there anything I need to provide in the child pipeline. File Reader snap I am using as shown in the previous screenshot.1.8KViews0likes3CommentsRe: Connect SFTP server to fetch multiple files
Hi Charlie, I’ve tried Directory Browser snap to fetch files. When I hook pipelineExecute Snap I am getting the error. Failure: Pipeline did not complete successfully, Reason: Snap errors: {ruuid=830fa0d3-2253-40b5-b419-2a4fd53dfa97, reason=Value referenced in the sub-expression ‘$Path’ is null, label=File Reader, resolution=Please check expression syntax and data types.}, Resolution: Fix the child pipeline errors and try again I might be giving the wrong expression in the parameter section. Can you please look into pipeline Execute and File reader snap screenshots and suggest the correction. Attached screenshots. Thanks, Amar1.8KViews0likes2CommentsRe: Connect SFTP server to fetch multiple files
Thank you, Charlie. I able to succeed with three snaps. Attached two screenshots for your ref. Can you please guide me how to extract data from the source file (it is the tab delimited file) to map to SQL table. How to extract the filename from the filepath. I want to extract a region from filename as well. Appreciate your support.2.2KViews0likes0CommentsRe: Connect SFTP server to fetch multiple files
Thank you, Charlie. I’ve followed your steps for the Multi File Reader Snap and able to succeed to fetch files from the remote directory. Attached screenshot for your ref. I also used BinaryToDocument snap and tried all options under encode or decode parameters. But I am getting below message in the dialogue window. “Preview data is not found or too large for browser to decrypt” Appreciate your next steps to get the document data. Thanks, Amar2.1KViews0likes0CommentsRe: Connect SFTP server to fetch multiple files
Hi Charlie, Thank you, for the giving other options. I am still struggling to process the files from the first snap. I am providing my findings for your reference. Appreciate your inputs further. MultiFileReader Snap is not taking sftp path under folder/file section. I am getting below exception: The same path is working other Snaps but not in MultiFileReader Snap. Provide your suggestions based on the error. Multi File Reader( Multi File Reader ) Reads binary data from an endpoint (e.g. file, sldb, s3, sftp, ftp, adl, etc). Snap has the following errostrong textrs. Please open the info box and edit the highlighted fields. Could not compile expression: _sftp://ftp.ext.hpe.com:2222/_SC … (Reason: Encountered extraneous input ‘:’ at line 1:4; expecting={, ‘||’, ‘&&’, ‘^’, ‘==’, ‘!=’, ‘>’, ‘<’, ‘>=’, ‘<=’, ‘+’, ‘-’, ‘*’, ‘/’, ‘%’, ‘?’, ‘instanceof’, ‘in’, ‘[’, ‘(’, ‘.’, PropertyRef}; Resolution: Please check expression syntax) I’ve tried DirectoryBrowser and able to fetch files from the SFTP server and used mapper snap to take filename into the variable. If I use file delete snap again it is throwing same error as multi-file reader snap exception. The file parameter is not taking SFTP path with variable filename. Appreciate your further support and also let me know how to extract data from the original file. The attributes from the directory browser snap showing bytes and name only. Thanks in advance, Amar3.3KViews0likes2CommentsRe: Connect SFTP server to fetch multiple files
Thank you, Charlie for the quick response. I’ve tried FilePoller and fetch four files from the remote SFTP directory. FilePoller output showing four different paths for each individual file. As a second snap, I have tried ForEach Snap but could not able to process the path. I might be giving wrong parameters. Attached ForEach Snap. Could you please guide how to process each individual file further. Thanks again Amar3.3KViews0likes0CommentsConnect SFTP server to fetch multiple files
Hi All, I am trying to connect SFTP server and fetch multiple files from the source remote directory. File Reader snap has an option to get SFTP account but doesn’t find how to provide remote directory path. In the remote directory, we are getting different types of files like *.txt where I need to apply filters based on the filenames to differentiate and invoke the different process to load into SQL DB. Once I fetch the files from remote location how to delete the files. Appreciate your inputs and any relevant links to look into more details. Let me know if you need any further details. Thanks in advance, Amar13KViews0likes16Comments