invalid value for POLARS_FORCE_CSV_INFER_READ_SIZE: {x}

Error message

invalid value for POLARS_FORCE_CSV_INFER_READ_SIZE: {x}

What it means

POLARS_FORCE_CSV_INFER_READ_SIZE is an internal knob (used mainly by polars' own tests) that pins a fixed chunk size for the streaming reader during CSV schema inference, overriding the adaptive initial_read_size. The value must parse as a NonZeroUsize; this panic fires inside the CompressedReader streaming loop when it does not. The variable is read on every call, so the panic hits as soon as inference streams the first rows.

Source

Thrown at crates/polars-io/src/csv/read/streaming.rs:409

/// Iterate over valid CSV lines produced by reader.
///
/// Returning `ConsumeDiscard` after `ConsumeKeep` is a logic error, since a segmented `Buffer`
/// can't be constructed.
fn for_each_line_from_reader_from_compressed_reader(
    parse_options: &CsvParseOptions,
    is_file_start: bool,
    mut prev_leftover: Buffer<u8>,
    initial_read_size: usize,
    reader: &mut CompressedReader,
    mut line_fn: impl FnMut(Buffer<u8>) -> PolarsResult<LineUse>,
) -> PolarsResult<Buffer<u8>> {
    let mut is_first_line = is_file_start;

    let fixed_read_size = std::env::var("POLARS_FORCE_CSV_INFER_READ_SIZE")
        .map(|x| {
            x.parse::<NonZeroUsize>()
                .unwrap_or_else(|_| {
                    panic!("invalid value for POLARS_FORCE_CSV_INFER_READ_SIZE: {x}")
                })
                .get()
        })
        .ok();

    let mut read_size = fixed_read_size.unwrap_or(initial_read_size);
    let mut retain_offset = None;

    loop {
        let (mut slice, bytes_read) = reader.read_next_slice(&prev_leftover, read_size)?;
        if slice.is_empty() {
            return Ok(Buffer::new());
        }

        if is_first_line {
            is_first_line = false;
            const UTF8_BOM_MARKER: Option<&[u8]> = Some(b"\xef\xbb\xbf");
            if slice.get(0..3) == UTF8_BOM_MARKER {

View on GitHub (pinned to 5d8ebabf11)

Solutions

unset POLARS_FORCE_CSV_INFER_READ_SIZE - inference then uses its normal adaptive sizing
Or set it to a positive byte count, e.g. export POLARS_FORCE_CSV_INFER_READ_SIZE=8192
Audit Dockerfiles/CI env blocks for leftover POLARS_FORCE_* debug variables

Example fix

# before
export POLARS_FORCE_CSV_INFER_READ_SIZE=0   # panics during read_csv inference

# after
unset POLARS_FORCE_CSV_INFER_READ_SIZE

Defensive patterns

Strategy: validation

Validate before calling

fn assert_no_debug_csv_env() -> Result<(), String> {
    if let Ok(s) = std::env::var("POLARS_FORCE_CSV_INFER_READ_SIZE") {
        if s.parse::<std::num::NonZeroUsize>().is_err() {
            return Err(format!("POLARS_FORCE_CSV_INFER_READ_SIZE must be a non-zero integer, got {s:?} - it is a polars-internal test knob, prefer unsetting it"));
        }
    }
    Ok(())
}

Try / catch

Deterministic env panic inside the inference loop; catch_unwind can only re-label it. Unset or fix the variable instead.

Prevention

Treat POLARS_FORCE_* as test-only: scrub them from production images
Assert on startup that no POLARS_FORCE_* variables are set
Keep debug env experiments in throwaway shells, never in shared .env files

When it happens

Trigger: The variable leaks into an environment (left over from debugging polars, copied from polars' test scripts or CI env) set to 0, a non-number, or a quoted/empty string, and any read_csv with schema inference runs through the compressed-reader path.

Common situations: Docker images or CI presets that baked debug POLARS_* variables; shell rc files carrying experiment leftovers; running with polars' own test env in a production container.

Understand the failure class

Background: "is not a valid" / "Invalid ... value" environment variable errors: how libraries validate env vars and what to do when they reject yours — this error's family across 48 libraries.

Related errors

AI-assisted analysis of pola-rs/polars@5d8ebabf11 (2026-08-19). Data as JSON: /api/errors/c055c88abd7c830c. Report an issue: GitHub.