invalid value for POLARS_IDEAL_SINK_MORSEL_SIZE_BYTES

Error message

invalid value for POLARS_IDEAL_SINK_MORSEL_SIZE_BYTES: {x}

What it means

POLARS_IDEAL_SINK_MORSEL_SIZE_BYTES optionally overrides the ideal byte size per sink morsel. It must parse as a non-zero u64; any other value (zero, non-numeric, empty) makes Polars panic while loading sink writer configuration.

Solutions

Set a plain positive integer byte count, e.g. export POLARS_IDEAL_SINK_MORSEL_SIZE_BYTES=8388608.
Unset the variable and rely on default sizing.
Do not use unit suffixes here unless your Polars version documents them; check docs for your version.

Example fix

# before
export POLARS_IDEAL_SINK_MORSEL_SIZE_BYTES=8MB
# after
export POLARS_IDEAL_SINK_MORSEL_SIZE_BYTES=8388608

Defensive patterns

Strategy: validation

Validate before calling

import os
v = os.environ.get("POLARS_IDEAL_SINK_MORSEL_SIZE_BYTES")
if v is not None:
    assert v.isdigit() and int(v) > 0, f"must be a plain positive integer byte count, got {v!r}"

Type guard

def is_plain_byte_count(s: str) -> bool:
    return s.isdigit() and int(s) > 0 and not any(c.isalpha() for c in s)

Try / catch

v = os.environ.get("POLARS_IDEAL_SINK_MORSEL_SIZE_BYTES", "")
if v and not v.isdigit():
    os.environ.pop("POLARS_IDEAL_SINK_MORSEL_SIZE_BYTES", None)

Prevention

Convert '8MB' style values to plain integers yourself (8 * 1024 * 1024)
No suffix parsing: set 8388608, not 8MB
Check both ROWS and BYTES vars together for consistency
Validate at process start before any sink runs

When it happens

Trigger: Setting POLARS_IDEAL_SINK_MORSEL_SIZE_BYTES to '0', '4MB' (raw u64 expected, not a size-suffixed string), or any unparsable text, then running a streaming sink; read in ideal_sink_morsel_size_env alongside the ROWS variant.

Common situations: Adding byte-size suffixes ('8m', '1GB') where a plain integer is required; conflicting/env collisions between ROWS and BYTES tuning vars.

Understand the failure class

Background: "is not a valid" / "Invalid ... value" environment variable errors: how libraries validate env vars and what to do when they reject yours — this error's family across 48 libraries.

Related errors

AI-assisted analysis of pola-rs/polars@fe841f959e (2026-09-18). Data as JSON: /api/errors/37123066bd0b164b. Report an issue: GitHub.

Appendix: source

Thrown at crates/polars-stream/src/nodes/io_sinks/writers/interface.rs:79

        Poll::Ready(file.map(|file| (file, self.sync_on_close)))
    }
}

/// Load ideal morsel size configuration from environment variables.
pub(super) fn ideal_sink_morsel_size_env() -> (Option<NonZeroIdxSize>, Option<NonZeroU64>) {
    let num_rows = std::env::var("POLARS_IDEAL_SINK_MORSEL_SIZE_ROWS")
        .map(|x| {
            x.parse::<NonZeroIdxSize>().ok().unwrap_or_else(|| {
                panic!("invalid value for POLARS_IDEAL_SINK_MORSEL_SIZE_ROWS: {x}")
            })
        })
        .ok();

    let num_bytes = std::env::var("POLARS_IDEAL_SINK_MORSEL_SIZE_BYTES")
        .map(|x| {
            x.parse::<NonZeroU64>().ok().unwrap_or_else(|| {
                panic!("invalid value for POLARS_IDEAL_SINK_MORSEL_SIZE_BYTES: {x}")
            })
        })
        .ok();

    (
        num_rows,
        num_bytes.or(num_rows.is_some().then_some(NonZeroU64::MAX)),
    )
}

View on GitHub (pinned to fe841f959e)