pola-rs/polars · error

invalid value for POLARS_ROW_GROUP_PREFETCH_KBYTES_BUDGET: {

Error message

invalid value for POLARS_ROW_GROUP_PREFETCH_KBYTES_BUDGET: {x}

What it means

Panic in ParquetReaderBuilder::set_execution_state: POLARS_ROW_GROUP_PREFETCH_KBYTES_BUDGET was set but is not a valid NonZeroUsize. The env var overrides the kilobyte memory bound for the parquet prefetch pipeline; the unparsable string {x} panics before streaming begins.

Source

Thrown at crates/polars-stream/src/nodes/io_sources/parquet/builder.rs:102

                    .max(get_request_budget() as usize)
                    .clamp(16, 2048),
            )
            .max(1);

        // Bound the max memory in use for the pipeline.
        // This should be large enough to be non-blocking, but small enough to avoid
        // excessive memory use from a run-away prefetch pipeline.
        // "Correct" formula: (a) max effective in-flight bdp-based bytes budget + (b) decode pipeline.
        // Since we do not know (a) at startup, we use  '3 * (b) + buffer' as a proxy for (a), where the
        // multiplier reflects the max gain factor for in-flight control.
        // TODO: Dynamically adapt the max memory to the observed BDP.
        // NOTE: This does not account for the decompression multiplier, so actual memory
        // usage can be substantially larger.
        let prefetch_kbytes_limit = std::env::var("POLARS_ROW_GROUP_PREFETCH_KBYTES_BUDGET")
            .map(|x| {
                x.parse::<NonZeroUsize>()
                    .unwrap_or_else(|_| {
                        panic!("invalid value for POLARS_ROW_GROUP_PREFETCH_KBYTES_BUDGET: {x}")
                    })
                    .get()
            })
            .unwrap_or({
                let target_chunk_size_kb = FetchConfig::random_access().chunk_size.div_ceil(1024);
                4 * execution_state.num_pipelines * target_chunk_size_kb
            })
            // Avoid deadlock.
            .max(polars_io::cloud::concurrency_config::get_download_chunk_size().div_ceil(1024));

        if config::verbose() {
            eprintln!(
                "[ParquetReaderBuilder]: prefetch_limit: {}, prefetch_kbytes_limit: {}",
                prefetch_limit, prefetch_kbytes_limit
            );
        }

        self.pipeline_budget

View on GitHub (pinned to 68506541d2)

Solutions

1. Set POLARS_ROW_GROUP_PREFETCH_KBYTES_BUDGET to a positive integer or unset it.

Example fix

export POLARS_ROW_GROUP_PREFETCH_KBYTES_BUDGET=131072

When it happens

Trigger: Invalid value in POLARS_ROW_GROUP_PREFETCH_KBYTES_BUDGET.

Common situations: See trigger scenarios.

AI-assisted analysis of pola-rs/polars@68506541d2 (2026-08-19). Data as JSON: /api/errors/b3a1d264ca3ed8c7. Report an issue: GitHub.