{"record":{"id":"dd545acf98eee57c","repo":"prestodb/presto","slug":"not-supported-dd545a","errorCode":"NOT_SUPPORTED","errorMessage":"GroupedKHyperLogLogState number of groups exceed limit %d set by khyperloglog-agg-group-limit","messagePattern":"GroupedKHyperLogLogState number of groups exceed limit (.+?) set by khyperloglog-agg-group-limit","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"error","filePath":"presto-main-base/src/main/java/com/facebook/presto/type/khyperloglog/KHyperLogLogStateFactory.java","lineNumber":90,"sourceCode":"        private long size;\n        private final long groupLimit;\n\n        public GroupedKHyperLogLogState(long groupLimit)\n        {\n            this.groupLimit = groupLimit;\n        }\n\n        @Override\n        public void setGroupId(long groupId)\n        {\n            this.groupId = groupId;\n        }\n\n        @Override\n        public void ensureCapacity(long size)\n        {\n            if (groupLimit > 0 && size > groupLimit) {\n                throw new PrestoException(NOT_SUPPORTED, format(\"GroupedKHyperLogLogState number of groups exceed limit %d set by khyperloglog-agg-group-limit\", groupLimit));\n            }\n            khlls.ensureCapacity(size);\n        }\n\n        @Override\n        public KHyperLogLog getKHLL()\n        {\n            return khlls.get(groupId);\n        }\n\n        @Override\n        public void setKHLL(KHyperLogLog value)\n        {\n            if (getKHLL() != null) {\n                size -= getKHLL().estimatedInMemorySize();\n            }\n            size += value.estimatedInMemorySize();\n            khlls.set(groupId, value);","sourceCodeStart":72,"sourceCodeEnd":108,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-main-base/src/main/java/com/facebook/presto/type/khyperloglog/KHyperLogLogStateFactory.java#L72-L108","documentation":"KHyperLogLog aggregations that group results into a GroupedKHyperLogLogState enforce a maximum number of distinct groups via the khyperloglog-agg-group-limit config. When the aggregation needs more groups than this limit, ensureCapacity throws NOT_SUPPORTED to bound memory use.","triggerScenarios":"Running a KHyperLogLog aggregate (khyperloglog_agg) with GROUP BY where the number of distinct group keys exceeds the configured khyperloglog-agg-group-limit at runtime.","commonSituations":"High-cardinality grouping columns (user IDs, URLs) with a cluster configured with a small conservative group limit; queries that worked on samples but explode on full data.","solutions":["Increase khyperloglog-agg-group-limit in the cluster configuration (or session property) to cover expected group cardinality.","Reduce cardinality of the GROUP BY key (coarser buckets, bucketing, pre-aggregation).","Split the query into batches or filter the input so groups stay under the limit."],"exampleFix":"// before (config.properties)\nkhyperloglog-agg-group-limit=10000\n// after\nkhyperloglog-agg-group-limit=1000000","handlingStrategy":"try-catch","validationCode":"// Before running the query, check expected group cardinality against the limit:\nlong distinctGroups = execute(\"SELECT approx_distinct(group_col) FROM src\").getLong(0);\nlong limit = getSessionProperty(\"khyperloglog-agg-group-limit\");\nboolean safe = distinctGroups <= limit;","typeGuard":null,"tryCatchPattern":"try (QueryResult r = runQuery(khllQuery)) {\n    // process results\n} catch (PrestoException e) {\n    if (e.getErrorCode() == NOT_SUPPORTED.toErrorCode() &&\n        e.getMessage().contains(\"khyperloglog-agg-group-limit\")) {\n        // raise the limit (SET SESSION khyperloglog_agg_group_limit = ...) and retry, or reduce group cardinality\n    } else {\n        throw e;\n    }\n}","preventionTips":["Set khyperloglog-agg-group-limit to comfortably exceed the max expected distinct group count.","Estimate group cardinality with approx_distinct before running KHLL aggregations.","Reduce GROUP BY cardinality (bucketing, coarser keys) for high-cardinality columns.","Prefer plain approx_distinct over khyperloglog_agg when per-group sketches are not needed."],"tags":["presto","khyperloglog","aggregation","memory-limit","configuration"],"backgroundTag":"group-limit-exceeded","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}