pola-rs/polars · error

invalid `POLARS_MAX_CACHED_METADATA_SCANS` value

Error message

invalid `POLARS_MAX_CACHED_METADATA_SCANS` value

What it means

Polars reads the environment variable POLARS_MAX_CACHED_METADATA_SCANS to cap how many scan sources keep cached metadata (default 8; 0 means unlimited). If the variable is set to something that does not parse as usize, `max_metadata_scan_cached` panics with this message.

Solutions

1. Set POLARS_MAX_CACHED_METADATA_SCANS to a non-negative integer (e.g. 8) or unset it to use the default
2. Use 0 to disable the cap (unlimited cached metadata scans)
3. Check container/k8s env definitions for typos or whitespace in the value

Example fix

# before
POLARS_MAX_CACHED_METADATA_SCANS=8.0
# after
POLARS_MAX_CACHED_METADATA_SCANS=8

import os
v = os.environ.get("POLARS_MAX_CACHED_METADATA_SCANS")
if v is not None:
    assert v.isdigit() or (v.startswith("-") and False), f"POLARS_MAX_CACHED_METADATA_SCANS must be a non-negative integer, got {v!r}"

def is_valid_scan_cache_value(v: str | None) -> bool:
    return v is None or v.isdigit()

When it happens

Trigger: Setting POLARS_MAX_CACHED_METADATA_SCANS to a non-integer or negative value (e.g. "many", "-1", "8.0", "") and then executing any lazy query that performs file scans (infer_or_parse path).

Common situations: Typo in a deployment config / .env file, shell quoting leaving stray characters, or copying a docs example with a wrong value.

Understand the failure class

Background: "is not a valid" / "Invalid ... value" environment variable errors: how libraries validate env vars and what to do when they reject yours — this error's family across 48 libraries.

Related errors

• in-flight byte budget init must be larger than the…
• integer
• integer
• invalid value for
• invalid value for POLARS_CLOUD_FOOTER_READ_SIZE

AI-assisted analysis of pola-rs/polars@fe841f959e (2026-09-18). Data as JSON: /api/errors/b9def59483564127. Report an issue: GitHub.

Appendix: source

Thrown at crates/polars-plan/src/plans/conversion/dsl_to_ir/scans.rs:762

        Some(PrimitiveLogicalType::Integer(
            IntegerType::UInt8 | IntegerType::UInt16 | IntegerType::UInt32 | IntegerType::UInt64
        ))
    ) || matches!(
        primitive_type.converted_type,
        Some(
            PrimitiveConvertedType::Uint8
                | PrimitiveConvertedType::Uint16
                | PrimitiveConvertedType::Uint32
                | PrimitiveConvertedType::Uint64
        )
    )
}

pub fn max_metadata_scan_cached() -> usize {
    static MAX_SCANS_METADATA_CACHED: LazyLock<usize> = LazyLock::new(|| {
        let value = std::env::var("POLARS_MAX_CACHED_METADATA_SCANS").map_or(8, |v| {
            v.parse::<usize>()
                .expect("invalid `POLARS_MAX_CACHED_METADATA_SCANS` value")
        });

        if value == 0 {
            return usize::MAX;
        }

        if polars_config::config().verbose() {
            eprintln!("parquet max cached metadata scans: {value}")
        };

        value
    });
    *MAX_SCANS_METADATA_CACHED
}

/// A file with at most this many record batches is counted exactly, a larger one
/// has its row count extrapolated from this many.
#[cfg(feature = "ipc")]

View on GitHub (pinned to fe841f959e)