cancel
Showing results for 
Search instead for 
Did you mean: 

HDFS Reader Glob Filter alone is not working

philliperamos
Contributor

Good day all,

I’m trying to use the HDFS Reader snap to read .csv data from a Azure Blob storage.
When I enter the specific filename shown in the screenshot below, I get data returned.
image

However, when I only have the glob filter (*.csv), no data is returned. See screenshot below.
image

Any idea as to why the glob filter alone is not working?
Thanks in advance for your assistance.

2 REPLIES 2

cstewart
Former Employee

I believe that the glob filter is only applied for the suggest operation in the Designer, not a run-time setting. If you want to scan for files, you may want to use the Hadoop directory Browser Snap, which will output the list of files, which you can then pass into the HDFS File Reader to do the actual file reading.

philliperamos
Contributor

Thanks @cstewart . Unfortunately we are not using Azure Data Lake for our infrastructure as at this point, we are using Blob storage. And the Hadoop Dir Browser needs an Azure Data Lake account.
We are considering moving to the data lake as it has other benefits that the organization can utilize.

Barring the Hadoop Dir browser snap, is there any other way?