What causes "HyperLogLog: Invalid bucket index" in risingwavelabs/risingwave?

Fix the rank computation to clamp/offset leading-zero counts into 1..=64 (e.g. `rank = zeros + 1`, capped at 64)

HyperLogLog: Invalid bucket index

Error message

HyperLogLog: Invalid bucket index

What it means

The append-only HyperLogLog bucket stores a single u8 register holding the leading-zero count for one hash bucket. Valid ranks are 1..=64; index 0 and indices above 64 are impossible outputs of the hash/rank computation, so receiving one indicates corrupted hashing logic or state and the update aborts.

Source

Thrown at src/expr/impl/src/aggregate/approx_count_distinct/append_only.rs:27

// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use risingwave_common::bail;
use risingwave_common_estimate_size::EstimateSize;
use risingwave_expr::Result;

use super::Bucket;

#[derive(Clone, Copy, Default, Debug, EstimateSize, PartialEq, Eq)]
pub struct AppendOnlyBucket(pub u8);

impl Bucket for AppendOnlyBucket {
    fn update(&mut self, index: u8, retract: bool) -> Result<()> {
        if index > 64 || index == 0 {
            bail!("HyperLogLog: Invalid bucket index");
        }
        if retract {
            bail!("HyperLogLog: Deletion in append-only bucket");
        }
        if index > self.0 {
            self.0 = index;
        }
        Ok(())
    }

    fn max(&self) -> u8 {
        self.0
    }
}

View on GitHub (pinned to 6469eb736d)

Solutions

Fix the rank computation to clamp/offset leading-zero counts into 1..=64 (e.g. `rank = zeros + 1`, capped at 64)
Verify the hash function width matches the HLL configuration (64-bit hashes expected)
Rebuild aggregation state if it came from an incompatible serialized format

Example fix

// before
let rank = hash.leading_zeros() as u8; // can be 0..=65
// after
let rank = ((hash.leading_zeros() as u8) + 1).min(64);

Defensive patterns

Strategy: validation

Validate before calling

fn valid_rank(index: u8) -> bool { (1..=64).contains(&index) }

Type guard

fn in_range(index: u8) -> bool { index != 0 && index <= 64 }

Try / catch

match bucket.update(index, false) {
    Err(e) if e.to_string().contains("Invalid bucket index") => return Err(anyhow!("hash rank out of range: {}", index)),
    other => other,
}

Prevention

Cap rank computation at 64 and offset leading_zeros by +1
Use the standard 64-bit hash (e.g. xxhash/ahash 64) for HLL
Add assertion tests on rank range in hash unit tests

When it happens

Trigger: `AppendOnlyBucket::update` called with `index == 0` or `index > 64` while feeding a new value into the approx_count_distinct aggregate.

Common situations: Custom/modified hash functions producing out-of-range ranks; memory corruption or deserialized aggregation state inconsistent with the current register width.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/a479ae452073de51. Report an issue: GitHub.