invalid value for POLARS_IDEAL_SINK_MORSEL_SIZE_ROWS

Error message

invalid value for POLARS_IDEAL_SINK_MORSEL_SIZE_ROWS: {x}

What it means

POLARS_IDEAL_SINK_MORSEL_SIZE_ROWS optionally overrides the ideal row count per morsel fed to sink writers. Polars parses it as NonZeroIdxSize; a value that is zero, negative, or not a valid size literal causes a panic during streaming sink setup.

Solutions

Set a positive integer row count, e.g. export POLARS_IDEAL_SINK_MORSEL_SIZE_ROWS=10000.
Unset the variable to use the default morsel sizing.
Check surrounding scripts for empty-string exports and strip whitespace/units.

Example fix

# before
export POLARS_IDEAL_SINK_MORSEL_SIZE_ROWS=0
# after
export POLARS_IDEAL_SINK_MORSEL_SIZE_ROWS=10000

Defensive patterns

Strategy: validation

Validate before calling

import os
v = os.environ.get("POLARS_IDEAL_SINK_MORSEL_SIZE_ROWS")
if v is not None:
    assert v.isdigit() and int(v) > 0, f"must be positive integer rows, got {v!r}"

Type guard

def is_positive_int(s: str) -> bool:
    return s.isdigit() and int(s) > 0

Try / catch

v = os.environ.get("POLARS_IDEAL_SINK_MORSEL_SIZE_ROWS", "")
if v and not v.isdigit():
    os.environ.pop("POLARS_IDEAL_SINK_MORSEL_SIZE_ROWS", None)  # use library default

Prevention

Only export these vars when the tuning value is actually present
Strip whitespace and units before exporting
Centralize POLARS_* tuning in one validated config module
Never default to empty string; default to unset

When it happens

Trigger: Setting POLARS_IDEAL_SINK_MORSEL_SIZE_ROWS to something like '0', 'abc', or a non-size token, then executing a sink (sink_parquet/sink_csv/...); read in ideal_sink_morsel_size_env, used by initialized_state and target_sink_morsel_size.

Common situations: Performance tuning sink morsel sizes from a runbook with a typo; scripting that exports empty strings when the tuning value is missing.

Understand the failure class

Background: "is not a valid" / "Invalid ... value" environment variable errors: how libraries validate env vars and what to do when they reject yours — this error's family across 48 libraries.

Related errors

AI-assisted analysis of pola-rs/polars@fe841f959e (2026-09-18). Data as JSON: /api/errors/16761428e6adef2d. Report an issue: GitHub.

Appendix: source

Thrown at crates/polars-stream/src/nodes/io_sinks/writers/interface.rs:71

    fn poll(
        mut self: std::pin::Pin<&mut Self>,
        cx: &mut std::task::Context<'_>,
    ) -> std::task::Poll<Self::Output> {
        use std::task::Poll;

        let file: Result<_, tokio::task::JoinError> = futures::ready!(self.handle.poll_unpin(cx));
        let file: PolarsResult<WriteTarget> = file.unwrap();

        Poll::Ready(file.map(|file| (file, self.sync_on_close)))
    }
}

/// Load ideal morsel size configuration from environment variables.
pub(super) fn ideal_sink_morsel_size_env() -> (Option<NonZeroIdxSize>, Option<NonZeroU64>) {
    let num_rows = std::env::var("POLARS_IDEAL_SINK_MORSEL_SIZE_ROWS")
        .map(|x| {
            x.parse::<NonZeroIdxSize>().ok().unwrap_or_else(|| {
                panic!("invalid value for POLARS_IDEAL_SINK_MORSEL_SIZE_ROWS: {x}")
            })
        })
        .ok();

    let num_bytes = std::env::var("POLARS_IDEAL_SINK_MORSEL_SIZE_BYTES")
        .map(|x| {
            x.parse::<NonZeroU64>().ok().unwrap_or_else(|| {
                panic!("invalid value for POLARS_IDEAL_SINK_MORSEL_SIZE_BYTES: {x}")
            })
        })
        .ok();

    (
        num_rows,
        num_bytes.or(num_rows.is_some().then_some(NonZeroU64::MAX)),
    )
}

View on GitHub (pinned to fe841f959e)