{"record":{"id":"b4f694306f7675f9","repo":"databendlabs/databend","slug":"sample-row-count-overflow","errorCode":null,"errorMessage":"sample row count overflow","messagePattern":"sample row count overflow","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/query/expression/src/sampler/fixed_size_sampler.rs","lineNumber":54,"sourceCode":"    pub fn new(k: usize, rng: R) -> Self {\n        let k = NonZeroUsize::new(k).expect(\"sample size must be greater than zero\");\n        Self {\n            samples: Vec::with_capacity(k.get()),\n            k: k.get(),\n            rows_seen: 0,\n            next_sample: None,\n            core: AlgoL::new(k, rng),\n        }\n    }\n\n    /// Consider one logical block while preserving the same result as one continuous row stream.\n    ///\n    /// `value_at` is evaluated only for rows entering the reservoir: every row during the initial\n    /// fill, then only the rows selected by Algorithm L.\n    pub fn add_block<F>(&mut self, rows: usize, mut value_at: F)\n    where F: FnMut(usize) -> T {\n        let start = self.rows_seen;\n        let end = start.checked_add(rows).expect(\"sample row count overflow\");\n        let mut row = 0;\n\n        if self.samples.len() < self.k {\n            let take = (self.k - self.samples.len()).min(rows);\n            self.samples.extend((0..take).map(&mut value_at));\n            row = take;\n\n            if self.samples.len() == self.k {\n                self.next_sample = (self.k - 1).checked_add(self.core.search());\n            }\n        }\n\n        while let Some(sample_index) = self.next_sample {\n            if sample_index >= end {\n                break;\n            }\n            debug_assert!(sample_index >= start + row);\n            row = sample_index - start;","sourceCodeStart":36,"sourceCodeEnd":72,"githubUrl":"https://github.com/databendlabs/databend/blob/288d84d76e20a2f8f7173bda9691eb6ece301aa9/src/query/expression/src/sampler/fixed_size_sampler.rs#L36-L72","documentation":"add_block tracks the cumulative number of rows seen (rows_seen) across blocks, and uses checked_add to detect overflow when adding the current block's row count. This panic fires when the running total of rows exceeds i64/usize range (or the internal representation), meaning the sampler has seen an impossible number of rows. It is an internal invariant guarding the reservoir statistics.","triggerScenarios":"Calling add_block repeatedly so that rows_seen + rows overflows usize/i64 — practically requires ~9 quintillion accumulated rows, e.g. rows value computed from corrupted offsets or a huge/negative-derived usize from a malformed block length.","commonSituations":"A bug upstream computing block row counts (e.g. wrong offset arithmetic producing a huge usize); corrupted column offsets causing a nonsense rows value rather than genuinely scanning that many rows.","solutions":["Audit the caller to ensure `rows` is a real row count derived from valid block offsets.","If legitimate huge inputs are expected, switch rows_seen to u128 or saturating arithmetic.","Return a proper error instead of expect when integrating with user-facing operators."],"exampleFix":"// before\nlet end = start.checked_add(rows).expect(\"sample row count overflow\");\n// after\nlet end = start.checked_add(rows).ok_or_else(|| ErrorCode::Internal(\"sample row count overflow\"))?;","handlingStrategy":"validation","validationCode":"debug_assert!(rows < usize::MAX / 2, \"implausible block row count: {rows}\");\nsampler.add_block(rows, value_at);","typeGuard":"fn plausible_row_count(rows: usize) -> bool { rows < usize::MAX / 2 }","tryCatchPattern":null,"preventionTips":["Derive block row counts from validated offsets (offsets[i+1] - offsets[i]) with sanity checks.","Never pass raw memory sizes as row counts."],"tags":["panic","overflow","reservoir-sampling"],"backgroundTag":"internal-invariant-violation","analyzedSha":"288d84d76e20a2f8f7173bda9691eb6ece301aa9","analyzedAt":"2026-09-11T11:29:36.208Z","contentChangedAt":"2026-09-11T11:29:36.208Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}