databendlabs/databend · error

sample row count overflow

Error message

sample row count overflow

What it means

add_block tracks the cumulative number of rows seen (rows_seen) across blocks, and uses checked_add to detect overflow when adding the current block's row count. This panic fires when the running total of rows exceeds i64/usize range (or the internal representation), meaning the sampler has seen an impossible number of rows. It is an internal invariant guarding the reservoir statistics.

Solutions

1. Audit the caller to ensure `rows` is a real row count derived from valid block offsets.
2. If legitimate huge inputs are expected, switch rows_seen to u128 or saturating arithmetic.
3. Return a proper error instead of expect when integrating with user-facing operators.

Example fix

// before
let end = start.checked_add(rows).expect("sample row count overflow");
// after
let end = start.checked_add(rows).ok_or_else(|| ErrorCode::Internal("sample row count overflow"))?;

debug_assert!(rows < usize::MAX / 2, "implausible block row count: {rows}");
sampler.add_block(rows, value_at);

fn plausible_row_count(rows: usize) -> bool { rows < usize::MAX / 2 }

When it happens

Trigger: Calling add_block repeatedly so that rows_seen + rows overflows usize/i64 — practically requires ~9 quintillion accumulated rows, e.g. rows value computed from corrupted offsets or a huge/negative-derived usize from a malformed block length.

Common situations: A bug upstream computing block row counts (e.g. wrong offset arithmetic producing a huge usize); corrupted column offsets causing a nonsense rows value rather than genuinely scanning that many rows.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• transfer target state offsets overflow
• transfer target initialized states overflow
• sample size must be greater than zero
• i256 overflow
• {}

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/b4f694306f7675f9. Report an issue: GitHub.

Appendix: source

Thrown at src/query/expression/src/sampler/fixed_size_sampler.rs:54

    pub fn new(k: usize, rng: R) -> Self {
        let k = NonZeroUsize::new(k).expect("sample size must be greater than zero");
        Self {
            samples: Vec::with_capacity(k.get()),
            k: k.get(),
            rows_seen: 0,
            next_sample: None,
            core: AlgoL::new(k, rng),
        }
    }

    /// Consider one logical block while preserving the same result as one continuous row stream.
    ///
    /// `value_at` is evaluated only for rows entering the reservoir: every row during the initial
    /// fill, then only the rows selected by Algorithm L.
    pub fn add_block<F>(&mut self, rows: usize, mut value_at: F)
    where F: FnMut(usize) -> T {
        let start = self.rows_seen;
        let end = start.checked_add(rows).expect("sample row count overflow");
        let mut row = 0;

        if self.samples.len() < self.k {
            let take = (self.k - self.samples.len()).min(rows);
            self.samples.extend((0..take).map(&mut value_at));
            row = take;

            if self.samples.len() == self.k {
                self.next_sample = (self.k - 1).checked_add(self.core.search());
            }
        }

        while let Some(sample_index) = self.next_sample {
            if sample_index >= end {
                break;
            }
            debug_assert!(sample_index >= start + row);
            row = sample_index - start;

View on GitHub (pinned to 288d84d76e)