How to get Running Totals


I want to get running totals: here is my raw dataset:

key1 key2 qtr ct

  1.  2.      1.  5
  2.  3.       2. 4
  3.  4.      3.  1
  4.  5.      4.   3

My result:

key1 key2 qtr RunningTotal

  1.  2.      1.  5
  2.  3.      2.  9
  3.  4.      3.  10
  4.  5.      4.  13


I have not tried it, but is it possible to set a pipeline parameter to zero and then using a mapper to increment the value?

Pipeline parameters are discussed here:

Not sure if there is a Snap that does this type of thing directly, you might need to use the Script snap.

Alternatively, you can use the GroupByN snap to collect all of the documents into a single big document, use the expression language to operate on that, and then split the docs out again.

Pipeline parameters cannot be changed during the execution of a pipeline.

I don’t think that will work since you can’t update a pipeline parameter. This is a tough one, you might end up doing this either on the source or target system.