pola-rs/polars · error

invalid value for POLARS_NDJSON_CHUNK_PREFETCH_LIMIT: {x}

Error message

invalid value for POLARS_NDJSON_CHUNK_PREFETCH_LIMIT: {x}

What it means

Panic in NDJsonReaderBuilder::set_execution_state: POLARS_NDJSON_CHUNK_PREFETCH_LIMIT was set but is not a valid NonZeroUsize (zero or non-numeric). The env var replaces the default NDJSON chunk prefetch count, so reader initialization aborts with the raw value {x}.

Source

Thrown at crates/polars-stream/src/nodes/io_sources/ndjson/builder.rs:59

#[cfg(feature = "json")]
impl FileReaderBuilder for NDJsonReaderBuilder {
    fn reader_name(&self) -> &str {
        "ndjson"
    }

    fn reader_capabilities(&self) -> ReaderCapabilities {
        ndjson_reader_capabilities()
    }

    fn set_execution_state(&self, execution_state: &crate::execute::StreamingExecutionState) {
        // The maximum number of chunks actively being prefetched at any given point in time.
        let prefetch_limit = std::env::var("POLARS_NDJSON_CHUNK_PREFETCH_LIMIT")
            .map(|x| {
                x.parse::<NonZeroUsize>()
                    .ok()
                    .unwrap_or_else(|| {
                        panic!("invalid value for POLARS_NDJSON_CHUNK_PREFETCH_LIMIT: {x}")
                    })
                    .get()
            })
            .unwrap_or(execution_state.num_pipelines.saturating_mul(2))
            .max(1);

        self.prefetch_limit.store(prefetch_limit);

        if config::verbose() {
            eprintln!(
                "[NDJsonReaderBuilder]: prefetch_limit: {}",
                self.prefetch_limit.load()
            );
        }

        self.prefetch_semaphore
            .set(Arc::new(tokio::sync::Semaphore::new(prefetch_limit)))
            .unwrap()

View on GitHub (pinned to 9b5d73fd00)

Solutions

1. Set POLARS_NDJSON_CHUNK_PREFETCH_LIMIT to a positive integer or unset it.

Example fix

export POLARS_NDJSON_CHUNK_PREFETCH_LIMIT=4

When it happens

Trigger: Invalid value in POLARS_NDJSON_CHUNK_PREFETCH_LIMIT.

Common situations: See trigger scenarios.

AI-assisted analysis of pola-rs/polars@9b5d73fd00 (2026-08-19). Data as JSON: /api/errors/7f26c5f6d65ae0f4. Report an issue: GitHub.