Group by fields with specific group size

If the reason you’re asking is a concern over memory use if the group sizes become too large, then I’d like to draw your attention to a new feature of the Group By Fields snap in our November (4.27) release. The snap has a new setting called Memory Sensitivity, a menu with two options:

• None: Same as the current behavior, where all consecutive documents with the same value for the grouping field(s) are placed together in a single output document. One output document per group.

• Dynamic: The snap may choose to split each group into multiple “parts”. So one output document per part rather than per group. In this mode, every output document contains a new “partInfo” object that contains metadata about that part, even for groups that only have a single part:

"partInfo" : {
      "groupSize" : 21,
      "numParts" : 5,
      "partIndex" : 1,
      "partSize" : 5
    }

When Dynamic mode is selected, the snap tracks statistics about available memory and the sizes of all groups processed so far during the pipeline’s execution to determine whether to split each new group. There’s also a new setting called Min. Part Size to specify a lower limit on part size when a group is split into multiple parts.

I realize this isn’t quite what you are asking for here, but I thought you should be aware of this feature.