pola-rs/polars · error

invalid value for POLARS_CSV_CHUNK_SIZE: {x}

Error message

invalid value for POLARS_CSV_CHUNK_SIZE: {x}

What it means

Environment-variable parsing guard in the streaming CSV source: the POLARS_CSV_CHUNK_SIZE override was set to a value that fails to parse as the expected unsigned integer type. The env var's raw value is the faulty input.

Source

Thrown at crates/polars-stream/src/nodes/io_sources/csv/mod.rs:227

            !(matches!(byte_source.as_ref(), &DynByteSource::Buffer(_)) && compression.is_none());

        const ASSUMED_COMPRESSION_RATIO: usize = 4;
        let decompressed_file_size_hint = match compression {
            None => Some(file_size),
            Some(_) => Some(file_size * ASSUMED_COMPRESSION_RATIO),
        };

        // Unify the two source options (uncompressed local file mmapp'ed, or streaming async with
        // transparent decompression), into one unified reader source.
        let reader_source = if use_async_prefetch {
            // Prepare parameters for Prefetch task.
            // TODO REFACTOR: use get_streaming_chunk_size(),
            const DEFAULT_CSV_CHUNK_SIZE: usize = 32 * 1024 * 1024;
            let memory_prefetch_func = get_memory_prefetch_func(verbose);
            let chunk_size = std::env::var("POLARS_CSV_CHUNK_SIZE")
                .map(|x| {
                    x.parse::<NonZeroUsize>()
                        .unwrap_or_else(|_| panic!("invalid value for POLARS_CSV_CHUNK_SIZE: {x}"))
                        .get()
                })
                .unwrap_or(DEFAULT_CSV_CHUNK_SIZE);

            let prefetch_limit = self
                .chunk_prefetch_sync
                .prefetch_limit
                .min(file_size.div_ceil(chunk_size))
                .max(1);

            let (prefetch_send, prefetch_recv) = tokio::sync::mpsc::channel(prefetch_limit);

            // Task: Prefetch.
            // Initiate parallel downloads of raw chunks.
            let byte_source = byte_source.clone();
            let prefetch_task = {
                let prefetch_semaphore = Arc::clone(&self.chunk_prefetch_sync.prefetch_semaphore);
                let prefetch_prev_all_spawned =

View on GitHub (pinned to 5d8ebabf11)

Solutions

1. Set POLARS_CSV_CHUNK_SIZE to a positive integer or unset it.

Example fix

export POLARS_CSV_CHUNK_SIZE=8192

When it happens

Trigger: Invalid value in POLARS_CSV_CHUNK_SIZE.

Common situations: See trigger scenarios.

AI-assisted analysis of pola-rs/polars@5d8ebabf11 (2026-08-19). Data as JSON: /api/errors/5212b9a85a5f9b64. Report an issue: GitHub.