pola-rs/polars · error

integer

Error message

integer

What it means

POLARS_PARQUET_DECODE_TARGET_VALUES_PER_THREAD is an escape-hatch env var (default 16_777_216) used when planning the new-streaming parquet decoder. It must parse as usize; a non-integer value panics via this expect while the pipeline is being constructed.

Source

Thrown at crates/polars-stream/src/nodes/io_sources/parquet/mod.rs:322

            });

            return Ok((rx, handle));
        }

        // Prepare parameters for dispatch
        let projected_arrow_fields = projected_arrow_fields()?.clone();
        let memory_prefetch_func = get_memory_prefetch_func(verbose);
        let row_group_prefetch_size = self
            .row_group_prefetch_sync
            .pipeline_budget
            .count_limit()
            .min(file_metadata.row_groups.len())
            .max(1);

        // This can be set to 1 to force column-per-thread parallelism, e.g. for bug reproduction.
        let target_values_per_thread =
            std::env::var("POLARS_PARQUET_DECODE_TARGET_VALUES_PER_THREAD")
                .map(|x| x.parse::<usize>().expect("integer").max(1))
                .unwrap_or(16_777_216);

        let is_full_projection = projected_arrow_fields.len() == file_schema.len();

        let (output_recv, handle) = ParquetReadImpl {
            projected_arrow_fields,
            is_full_projection,
            predicate,
            // TODO: Refactor to avoid full clone
            options: Arc::unwrap_or_clone(self.config.clone()),
            byte_source,
            normalized_pre_slice: normalized_pre_slice.map(|x| match x {
                Slice::Positive { offset, len } => (offset, len),
                Slice::Negative { .. } => unreachable!(),
            }),
            metadata: file_metadata,
            config: io_sources::parquet::Config {
                num_pipelines,

View on GitHub (pinned to df599052da)

Solutions

1. Set a plain positive integer: export POLARS_PARQUET_DECODE_TARGET_VALUES_PER_THREAD=16777216
2. Unset the variable to use the default (16_777_216)
3. Remove the variable once debugging is finished so normal planning resumes
4. Audit shell profiles, docker ENV, and CI secrets for stale tuning variables

Example fix

# before
export POLARS_PARQUET_DECODE_TARGET_VALUES_PER_THREAD=1M  # panic: 'integer'

# after
export POLARS_PARQUET_DECODE_TARGET_VALUES_PER_THREAD=1048576
# or: unset POLARS_PARQUET_DECODE_TARGET_VALUES_PER_THREAD

# bootstrap validation for numeric tuning knobs
import os
v = os.environ.get("POLARS_PARQUET_DECODE_TARGET_VALUES_PER_THREAD")
if v is not None and not v.isdigit():
    raise ValueError("POLARS_PARQUET_DECODE_TARGET_VALUES_PER_THREAD must be a plain integer")

When it happens

Trigger: Setting the variable to a non-numeric or negative value (e.g. '1e6', '1M', '-1', '') and then executing a streaming parquet read (engine='streaming') whose plan setup reads the variable.

Common situations: Performance-tuning knobs copied from blog posts with units attached; debug configurations left over from bug reproduction (the comment says it is used to force column-per-thread parallelism); CI environments with stale values.

Related errors

• invalid `POLARS_MAX_CACHED_METADATA_SCANS` value
• invalid POLARS_REGEX_SIZE_LIMIT
• merge-join kernel not implemented for {:?}
• unimplemented: AnonymousScan
• integer

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/752b0a29b7bce462. Report an issue: GitHub.