invalid value for POLARS_CLOUD_WRITER_COPY_BUFFER_SIZE: {s}

Error message

invalid value for POLARS_CLOUD_WRITER_COPY_BUFFER_SIZE: {s}

What it means

cloud_writer_copy_buffer_size() sizes the staging copy buffer for cloud uploads from POLARS_CLOUD_WRITER_COPY_BUFFER_SIZE, defaulting to 16 MiB. The string must parse as a NonZeroUsize, so 0 and any non-numeric value panic; unlike the coalesce knob there is no >= 2 filter, so 1 is technically accepted. The LazyLock means the panic happens on the first cloud write that touches the buffer, not when the variable is set.

Source

Thrown at crates/polars-io/src/configs.rs:73

        }

        if polars_core::config::verbose() {
            eprintln!("cloud_writer coalesce_run_length: {v}")
        }

        v
    });
}

pub(crate) fn cloud_writer_copy_buffer_size() -> NonZeroUsize {
    return *COPY_BUFFER_SIZE;

    static COPY_BUFFER_SIZE: LazyLock<NonZeroUsize> = LazyLock::new(|| {
        let mut v: NonZeroUsize = const { NonZeroUsize::new(16 * 1024 * 1024).unwrap() };

        if let Ok(s) = std::env::var("POLARS_CLOUD_WRITER_COPY_BUFFER_SIZE") {
            v = s.parse::<NonZeroUsize>().unwrap_or_else(|_| {
                panic!("invalid value for POLARS_CLOUD_WRITER_COPY_BUFFER_SIZE: {s}")
            })
        }

        if polars_core::config::verbose() {
            eprintln!("cloud_writer copy_buffer_size: {v}")
        }

        v
    });
}

View on GitHub (pinned to 9b5d73fd00)

Solutions

Set a plain positive integer of bytes, e.g. export POLARS_CLOUD_WRITER_COPY_BUFFER_SIZE=16777216
Or unset the variable to use the 16 MiB default
Remember 0 is impossible by design (NonZeroUsize) - to lower memory pick a smaller positive size such as 1048576

Example fix

# before (0 is rejected by NonZeroUsize)
export POLARS_CLOUD_WRITER_COPY_BUFFER_SIZE=0

# after
export POLARS_CLOUD_WRITER_COPY_BUFFER_SIZE=16777216   # 16 MiB
# or: unset POLARS_CLOUD_WRITER_COPY_BUFFER_SIZE

Defensive patterns

Strategy: validation

Validate before calling

fn validate_polars_env() -> Result<(), String> {
    if let Ok(s) = std::env::var("POLARS_CLOUD_WRITER_COPY_BUFFER_SIZE") {
        if s.parse::<std::num::NonZeroUsize>().is_err() {
            return Err(format!(
                "POLARS_CLOUD_WRITER_COPY_BUFFER_SIZE must be a non-zero integer of bytes, got {s:?}"
            ));
        }
    }
    Ok(())
}
// call once at startup, before the first cloud write

Try / catch

Deterministic LazyLock panic: wrap the first cloud write in catch_unwind only to convert it into a clean startup error; the real fix is correcting or unsetting the variable.

Prevention

Never set 0 for buffer knobs - check each knob's type (usize vs NonZeroUsize) before tuning
Template env values through one validated config module
Add a smoke cloud-write to CI with the production env vars

When it happens

Trigger: Setting POLARS_CLOUD_WRITER_COPY_BUFFER_SIZE=0 (a common attempt to 'disable' buffering to save memory), a value with a unit suffix ("16M"), scientific notation ("1e6"), or any non-number, then triggering the first cloud write that copies data through the buffer.

Common situations: Memory-tuning attempts on containers with tight limits; templates that mangle numbers (quotes, spaces, suffixes); values copied from docs of a different tool.

Understand the failure class

Background: "is not a valid" / "Invalid ... value" environment variable errors: how libraries validate env vars and what to do when they reject yours — this error's family across 48 libraries.

Related errors

AI-assisted analysis of pola-rs/polars@9b5d73fd00 (2026-08-19). Data as JSON: /api/errors/32a37755204199f0. Report an issue: GitHub.