{"record":{"id":"0cc86871e411c143","repo":"risingwavelabs/risingwave","slug":"hyperloglog-count-exceeds-maximum-bucket-value-yo","errorCode":null,"errorMessage":"HyperLogLog: Count exceeds maximum bucket value.Your data stream may have too many repeated values or too large acardinality for approx_count_distinct to handle (max: 2^64 - 1)","messagePattern":"HyperLogLog: Count exceeds maximum bucket value\\.Your data stream may have too many repeated values or too large acardinality for approx_count_distinct to handle \\(max: 2\\^64 - 1\\)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/expr/impl/src/aggregate/approx_count_distinct/updatable.rs","lineNumber":132,"sourceCode":"            sparse_counts: SparseCount::new(),\n        }\n    }\n}\n\nimpl<const DENSE_BITS: usize> Bucket for UpdatableBucket<DENSE_BITS> {\n    fn update(&mut self, index: u8, retract: bool) -> Result<()> {\n        if index > 64 || index == 0 {\n            bail!(\"HyperLogLog: Invalid bucket index\");\n        }\n\n        let count = self.get_bucket(index)?;\n\n        if !retract {\n            if index > DENSE_BITS as u8 {\n                self.sparse_counts.add(index);\n            } else if index >= 1 {\n                if count == u64::MAX {\n                    bail!(\n                        \"HyperLogLog: Count exceeds maximum bucket value.\\\n                        Your data stream may have too many repeated values or too large a\\\n                        cardinality for approx_count_distinct to handle (max: 2^64 - 1)\"\n                    );\n                }\n                self.dense_counts[index as usize - 1] = count + 1;\n            }\n        } else {\n            // We don't have to worry about the user deleting nonexistent elements, so the counts\n            // can never go below 0.\n            if index > DENSE_BITS as u8 {\n                self.sparse_counts.subtract(index);\n            } else if index >= 1 {\n                self.dense_counts[index as usize - 1] = count - 1;\n            }\n        }\n\n        Ok(())","sourceCodeStart":114,"sourceCodeEnd":150,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/expr/impl/src/aggregate/approx_count_distinct/updatable.rs#L114-L150","documentation":"Each dense bucket in the updatable HyperLogLog stores its counter in a u64. When a non-retract update tries to increment a bucket already at u64::MAX, the addition would overflow, so the library refuses instead of silently wrapping. This implies an astronomically large number of repeated register hits (max cardinality 2^64 - 1), so it practically indicates corrupted or pathological state.","triggerScenarios":"Calling `update(index, retract=false)` on a bucket whose current count equals u64::MAX; i.e. the same index incremented 2^64-1 times or a bucket pre-loaded with u64::MAX via merge/restore.","commonSituations":"Corrupted aggregation state in a materialized view; a merge of sketches with poisoned registers; extremely long-lived streaming jobs accumulating in one bucket (theoretical only).","solutions":["Rebuild the materialized view / aggregation state to reset bucket counters.","Check for state corruption from crash recovery or version migration and re-backfill the aggregation.","File an issue if this reproduces with normal data volumes; real streams cannot legitimately hit this bound."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// before incrementing, check the bucket\nif !retract && bucket_count == u64::MAX {\n    return Err(anyhow!(\"bucket {} saturated\", index));\n}","typeGuard":"fn can_increment(count: u64) -> bool { count < u64::MAX }","tryCatchPattern":null,"preventionTips":["Treat this error as a state-corruption signal; rebuild the MV.","Monitor cardinality estimates; real workloads cannot saturate u64 buckets.","Validate merged sketch registers before applying updates."],"tags":["rust","hyperloglog","overflow","aggregation-state"],"backgroundTag":"value-out-of-range","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}