pola-rs/polars · error

invalid value for POLARS_FORCE_NDJSON_READ_SIZE: {x}

Error message

invalid value for POLARS_FORCE_NDJSON_READ_SIZE: {x}

What it means

Panic in LineBatchProducer::new: POLARS_FORCE_NDJSON_READ_SIZE was set but failed to parse as a NonZeroUsize. This debug/override env var fixes the byte read size used by the NDJSON line batcher, so an invalid value {x} panics during producer construction.

Source

Thrown at crates/polars-stream/src/nodes/io_sources/ndjson/line_batch_distributor.rs:141

    prev_leftover: Buffer<u8>,
    read_size: usize,
    chunk_idx: usize,
    finished: bool,
}

impl LineBatchProducer {
    fn new(
        mut reader: ByteSourceReader<ReaderSource>,
        reverse: bool,
        row_skipper: RowSkipper,
        uncompressed_file_size_hint: Option<usize>,
        use_prefetch_l2: bool,
        verbose: bool,
    ) -> PolarsResult<Self> {
        let fixed_read_size = std::env::var("POLARS_FORCE_NDJSON_READ_SIZE")
            .map(|x| {
                x.parse::<NonZeroUsize>().ok().unwrap_or_else(|| {
                    panic!("invalid value for POLARS_FORCE_NDJSON_READ_SIZE: {x}")
                })
            })
            .ok();

        let read_size = fixed_read_size
            .map(NonZeroUsize::get)
            .unwrap_or_else(ByteSourceReader::<ReaderSource>::initial_read_size);

        if verbose {
            eprintln!(
                "[NDJson LineBatchDistributor]: \
                n_rows_to_skip: {}, \
                reverse: {reverse}, \
                fixed_read_size: {fixed_read_size:?}",
                row_skipper.cfg_n_rows_to_skip,
            );
        }

View on GitHub (pinned to 9b5d73fd00)

Solutions

1. Set POLARS_FORCE_NDJSON_READ_SIZE to a positive integer or unset it.

Example fix

export POLARS_FORCE_NDJSON_READ_SIZE=16384

When it happens

Trigger: Invalid value in POLARS_FORCE_NDJSON_READ_SIZE.

Common situations: See trigger scenarios.

AI-assisted analysis of pola-rs/polars@9b5d73fd00 (2026-08-19). Data as JSON: /api/errors/22dc1f3a279a55a2. Report an issue: GitHub.