Forum Discussion

mongley's avatar
mongley
New Contributor II
8 years ago
Solved

Drop first record

Hi All,

I have a variable number of records coming from a mapper and will always want to drop the first. Is there an easy way to do this?

Thanks,
Mark

  • tstack's avatar
    tstack
    8 years ago

    If you need to drop records, the Filter snap is usually a good option. In this case, you can use the snap.in.totalCount variable to get the total number of records that have been read by the snap. So, to skip the first one, you can use the following for the Filter Expression in the snap configuration:

    snap.in.totalCount != 1

6 Replies

  • tstack's avatar
    tstack
    Former Employee
    8 years ago

    If you need to drop records, the Filter snap is usually a good option. In this case, you can use the snap.in.totalCount variable to get the total number of records that have been read by the snap. So, to skip the first one, you can use the following for the Filter Expression in the snap configuration:

    snap.in.totalCount != 1
  • christwr's avatar
    christwr
    Contributor III
    8 years ago

    There are some Head/Tail snaps you might be able to use. Perhaps a Tail snap with an offset of 1.

    • mongley's avatar
      mongley
      New Contributor II
      8 years ago

      Thanks, I have taken a look at Head and Tail. They look like they would do the job however I need to know the number of documents to be processed - which I don’t. Is there a method that would return this?

  • christwr's avatar
    christwr
    Contributor III
    8 years ago

    There is an Aggregate snap, which you do stuff like COUNT, but not sure it’s worth it. If you did a Head snap with Offset=1 and Documents=1000000 (or some arbitrary max) would that work?