Good day all,
I’m trying to use the HDFS Reader snap to read .csv data from a Azure Blob storage.
When I enter the specific filename shown in the screenshot below, I get data returned.
However, when I only have the glob filter (*.csv), no data is returned. See screenshot below.
Any idea as to why the glob filter alone is not working?
Thanks in advance for your assistance.
I believe that the glob filter is only applied for the suggest operation in the Designer, not a run-time setting. If you want to scan for files, you may want to use the Hadoop directory Browser Snap, which will output the list of files, which you can then pass into the HDFS File Reader to do the actual file reading.
Thanks @cstewart . Unfortunately we are not using Azure Data Lake for our infrastructure as at this point, we are using Blob storage. And the Hadoop Dir Browser needs an Azure Data Lake account.
We are considering moving to the data lake as it has other benefits that the organization can utilize.
Barring the Hadoop Dir browser snap, is there any other way?
