pola-rs/polars · error

invalid value for POLARS_MAX_CONCURRENT_SCANS: {x}

Error message

invalid value for POLARS_MAX_CONCURRENT_SCANS: {x}

What it means

Panic in calc_max_concurrent_scans: POLARS_MAX_CONCURRENT_SCANS was set but cannot be parsed as a usize. The env var caps how many files multi-scan reads in parallel; the malformed string {x} aborts scan planning.

Source

Thrown at crates/polars-stream/src/nodes/io_sources/multi_scan/functions/mod.rs:46

    let max_files_with_slice = match pre_slice {
        // Calculate the max number of files assuming 1 row per file.
        Some(v @ Slice::Positive { .. }) => v.end_position().max(1),
        Some(Slice::Negative { .. }) | None => usize::MAX,
    };

    // Set this generously high, there are users who scan 10,000's of small files from the cloud.
    num_pipelines
        .saturating_add(3)
        .min(max_files_with_slice)
        .min(num_sources)
        .clamp(1, 128)
}

pub fn calc_max_concurrent_scans(num_pipelines: usize, num_sources: usize) -> usize {
    if let Ok(v) = std::env::var("POLARS_MAX_CONCURRENT_SCANS").map(|x| {
        x.parse::<NonZeroUsize>()
            .unwrap_or_else(|_| panic!("invalid value for POLARS_MAX_CONCURRENT_SCANS: {x}"))
            .get()
    }) {
        return v;
    }

    num_pipelines.min(num_sources).clamp(1, 128)
}

pub async fn is_compressed_source(
    scan_source: ScanSource,
    cloud_options: Option<Arc<CloudOptions>>,
    io_metrics: Option<Arc<IOMetrics>>,
) -> PolarsResult<bool> {
    let byte_source_builder = if scan_source.is_cloud_url() || polars_config::config().force_async()
    {
        DynByteSourceBuilder::ObjectStore(FetchConfig::legacy())
    } else {
        DynByteSourceBuilder::Mmap

View on GitHub (pinned to 9b5d73fd00)

Solutions

1. Set POLARS_MAX_CONCURRENT_SCANS to a positive integer or unset it.

Example fix

export POLARS_MAX_CONCURRENT_SCANS=4

When it happens

Trigger: Invalid value in POLARS_MAX_CONCURRENT_SCANS.

Common situations: See trigger scenarios.

AI-assisted analysis of pola-rs/polars@9b5d73fd00 (2026-08-19). Data as JSON: /api/errors/35035b8dc986474d. Report an issue: GitHub.