Forum Discussion

dgetchius's avatar
dgetchius
New Contributor
5 years ago

Record Count per file

Hi, I am reading multiple csv files from a SFTP location. I need to get the # of records per file, so I can put that info in a summary email. Any ideas on how I can retrieve the filename and # of records for each file?

Thanks!
Dawn

aggregate
pipeline

1 Reply

  • Supratim's avatar
    Supratim
    Contributor III
    5 years ago

    @dgetchius You can refer below design pattern, which will help you to identify the name of the file you process through dashboard.

    1. Parent pipeline: Use directory browser to pull all filename details and pass $Path to child pipeline parameter.
    2. Child Pipeline: Read file using pipeline param (Path) → use csv parser → copy (one for count and one for your business process)–> aggregator snap (at count flow)–> mapper( to capture count and use pipeline param for getting file name).

    If you want to complete all logic in one pipeline, then can use Binary to Document snap after reading file can give you filename details as well.