{"record":{"id":"9a09ff7dc88c42cb","repo":"apache/hadoop","slug":"too-many-counter-groups-size-max-groupsmax","errorCode":null,"errorMessage":"Too many counter groups: ${size} max=${groupsMax}","messagePattern":"Too many counter groups: (.+?) max=(.+?)","errorType":"validation","errorClass":"LimitExceededException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/Limits.java","lineNumber":114,"sourceCode":"    if (firstViolation != null) {\n      throw new LimitExceededException(firstViolation);\n    }\n    int countersMax = getCountersMax();\n    if (size > countersMax) {\n      firstViolation = new LimitExceededException(\"Too many counters: \"+ size +\n                                                  \" max=\"+ countersMax);\n      throw firstViolation;\n    }\n  }\n\n  public synchronized void incrCounters() {\n    checkCounters(totalCounters + 1);\n    ++totalCounters;\n  }\n\n  public synchronized void checkGroups(int size) {\n    if (firstViolation != null) {\n      throw new LimitExceededException(firstViolation);\n    }\n    int groupsMax = getGroupsMax();\n    if (size > groupsMax) {\n      firstViolation = new LimitExceededException(\"Too many counter groups: \"+\n                                                  size +\" max=\"+ groupsMax);\n    }\n  }\n\n  public synchronized LimitExceededException violation() {\n    return firstViolation;\n  }\n}\n","sourceCodeStart":96,"sourceCodeEnd":127,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/Limits.java#L96-L127","documentation":"Limits.checkGroups enforces mapreduce.job.counters.groups.max (default 50, MRJobConfig.COUNTER_GROUPS_MAX_DEFAULT). Unlike checkCounters, this site does not throw: it only records a LimitExceededException(\"Too many counter groups\") in firstViolation. The cached violation is thrown later by the next checkCounters()/incrCounters() call (or read via Limits.violation()), so the stack trace you see is usually at a counters increment, with this message as the cause.","triggerScenarios":"Registering more distinct counter group names than mapreduce.job.counters.groups.max - e.g. getGroup(\"group-\" + i) for many i - followed by any later counter add, which rethrows the cached groups violation. Also triggered during Counters deserialization (readFields calls checkGroups while loading groups).","commonSituations":"Dynamic group names from user code (per-module/per-tenant groups); aggregation of many libraries' counters; jobs migrated to clusters with default (50) group limit after using more on another cluster; history/audit tooling parsing large counters.","solutions":["Raise mapreduce.job.counters.groups.max (and mapreduce.job.counters.max) in the job configuration to cover your real group count.","Consolidate dynamic groups into one group with structured counter names.","Audit group-name cardinality the same way as counter-name cardinality.","Set the limits via Limits.init(conf) with the intended Configuration before any Counters object is created in the JVM (Limits are static-once)."],"exampleFix":"// before: one group per tenant\ncontext.getCounter(\"tenant-\" + tenantId, \"records\").increment(1);\n\n// after: one group, structured names\ncontext.getCounter(\"tenant-records\", \"TENANT\" + tenantId).increment(1);\n// and/or raise the cap:\nconf.setInt(\"mapreduce.job.counters.groups.max\", 200);","handlingStrategy":"validation","validationCode":"int groupsMax = conf.getInt(\"mapreduce.job.counters.groups.max\", 50);\nSet<String> plannedGroups = /* enumerate group names your job emits */;\nif (plannedGroups.size() > groupsMax) {\n  throw new IllegalStateException(\"Job emits \" + plannedGroups.size()\n      + \" counter groups; raise mapreduce.job.counters.groups.max or consolidate\");\n}","typeGuard":null,"tryCatchPattern":"catch (LimitExceededException e) with cause/message \"Too many counter groups\": the violation was recorded earlier by checkGroups - reduce group count and rebuild the Counters object; retrying on the same object keeps rethrowing.","preventionTips":["Keep the group set small and fixed; encode dynamic dimensions in counter names inside one group.","Configure mapreduce.job.counters.groups.max explicitly when you legitimately need more than 50 groups.","Remember the throw is deferred: the stack trace appears at a later counter operation, not at group creation."],"tags":["hadoop","mapreduce","counters","limits","configuration","deferred-throw"],"backgroundTag":"counter-limit-exceeded","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}