pola-rs/polars · error

subtraction overflow

Error message

subtraction overflow: {self:?} - {other:?}

What it means

RowCounter::sub subtracts another RowCounter (physical_rows and deleted_rows counters) field-by-field and panics if any field would underflow below zero. The library treats a negative row count as an unrecoverable logic error, so it panics instead of returning a wrapped/checked value.

Solutions

1. Verify the subtrahend RowCounter is derived from the same (or a smaller) row range as the minuend
2. Add checked_sub / debug assertions upstream to catch the inconsistent slice before sub() is reached
3. Print both RowCounters (they implement Debug) to see which field overflows and fix the producing code
4. Upgrade polars: similar row-counting bugs are often fixed in newer releases

Example fix

// before
let remaining = total.sub(consumed);
// after
let remaining = if total.physical_rows >= consumed.physical_rows && total.deleted_rows >= consumed.deleted_rows {
    total.sub(consumed)
} else {
    RowCounter { physical_rows: 0, deleted_rows: 0 } // or handle defensively
};

fn can_sub(total: RowCounter, consumed: RowCounter) -> bool {
    total.physical_rows >= consumed.physical_rows && total.deleted_rows >= consumed.deleted_rows
}

When it happens

Trigger: Calling sub() with an `other` RowCounter whose physical_rows or deleted_rows exceeds the receiver's, e.g. resolve_negative_slice subtracting a larger negative-slice row count from a smaller counter.

Common situations: Slice arithmetic bugs where a negative slice's start exceeds the row count; mis-estimated chunk boundaries in parallel engines; using a RowCounter accumulated over a different range than the one being subtracted.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• cannot coerce datatypes
• length to fit in `usize`
• activate dtype
• adding decimals with different precisions/scales is not…
• adding durations with different units is not supported here

AI-assisted analysis of pola-rs/polars@fe841f959e (2026-09-18). Data as JSON: /api/errors/163072ee81af92b8. Report an issue: GitHub.

Appendix: source

Thrown at crates/polars-utils/src/row_counter.rs:106

                // `num_rows()` will start to decrease).
                physical_rows?
            };

            Some(Self {
                physical_rows,
                deleted_rows,
            })
        })()
        .unwrap_or_else(|| panic!("addition overflow: {self:?} + {other:?}"))
    }

    /// # Panics
    /// Panics if subtraction overflows.
    #[allow(clippy::should_implement_trait)]
    pub fn sub(self, other: Self) -> Self {
        let func = |a: usize, b: usize| {
            a.checked_sub(b)
                .unwrap_or_else(|| panic!("subtraction overflow: {self:?} - {other:?}"))
        };

        Self {
            physical_rows: func(self.physical_rows, other.physical_rows),
            deleted_rows: func(self.deleted_rows, other.deleted_rows),
        }
    }

    /// Returns the number of rows after applying deletions. This returns an
    /// error if there are more deleted rows than physical rows.
    pub fn num_rows(&self) -> PolarsResult<usize> {
        self.physical_rows
            .checked_sub(self.deleted_rows)
            .ok_or_else(|| {
                polars_err!(
                    ComputeError: "RowCounter: Invalid state: \
                    number of rows removed by deletion files ({}) \
                    is greater than the number of rows physically present ({})",

View on GitHub (pinned to fe841f959e)