This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
FAQ
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

HDFS Reader Glob Filter alone is not working

philliperamos
Contributor

‎09-26-2019 05:59 AM

Good day all,

I’m trying to use the HDFS Reader snap to read .csv data from a Azure Blob storage.
When I enter the specific filename shown in the screenshot below, I get data returned.
image

However, when I only have the glob filter (*.csv), no data is returned. See screenshot below.
image

Any idea as to why the glob filter alone is not working?
Thanks in advance for your assistance.

0 Kudos
Reply
2 REPLIES 2

cstewart
Former Employee

‎09-26-2019 09:07 AM

I believe that the glob filter is only applied for the suggest operation in the Designer, not a run-time setting. If you want to scan for files, you may want to use the Hadoop directory Browser Snap, which will output the list of files, which you can then pass into the HDFS File Reader to do the actual file reading.

0 Kudos
Reply

philliperamos
Contributor

‎09-26-2019 11:04 AM

Thanks @cstewart . Unfortunately we are not using Azure Data Lake for our infrastructure as at this point, we are using Blob storage. And the Hadoop Dir Browser needs an Azure Data Lake account.
We are considering moving to the data lake as it has other benefits that the organization can utilize.

Barring the Hadoop Dir browser snap, is there any other way?

0 Kudos
Reply