What causes "Invalid `POLARS_PQ_PREFILTERED_MASK` value '{v}'." in pola-rs/polars?

Use one of the exact lowercase literals: export POLARS_PQ_PREFILTERED_MASK=pre (or auto|post)

Invalid `POLARS_PQ_PREFILTERED_MASK` value '{v}'.

Error message

Invalid `POLARS_PQ_PREFILTERED_MASK` value '{v}'.

What it means

PrefilterMaskSetting::init_from_env() maps POLARS_PQ_PREFILTERED_MASK to the strategy for applying row-group masks during predicate pushdown in Parquet reads: exactly "auto" (cost heuristic, default), "pre" (push mask before decode) or "post". Any other string - including case variants like "Pre" - panics the first time the setting is initialized during a Parquet scan with a predicate.

Source

Thrown at crates/polars-io/src/parquet/read/read_impl.rs:515

    // Closer to 0: pre-filtering is probably better.
    // Closer to 1: post-filtering is probably better.
    (num_edges / rg_len).clamp(0.0, 1.0)
}

#[derive(Clone, Copy)]
pub enum PrefilterMaskSetting {
    Auto,
    Pre,
    Post,
}

impl PrefilterMaskSetting {
    pub fn init_from_env() -> Self {
        std::env::var("POLARS_PQ_PREFILTERED_MASK").map_or(Self::Auto, |v| match &v[..] {
            "auto" => Self::Auto,
            "pre" => Self::Pre,
            "post" => Self::Post,
            _ => panic!("Invalid `POLARS_PQ_PREFILTERED_MASK` value '{v}'."),
        })
    }

    pub fn should_prefilter(&self, prefilter_cost: f64, dtype: &ArrowDataType) -> bool {
        match self {
            Self::Auto => {
                // Prefiltering is only expensive for nested types so we make the cut-off quite
                // high.
                let is_nested = dtype.is_nested();

                // We empirically selected these numbers.
                !is_nested && prefilter_cost <= 0.01
            },
            Self::Pre => true,
            Self::Post => false,
        }
    }
}

View on GitHub (pinned to fc24390824)

Solutions

Use one of the exact lowercase literals: export POLARS_PQ_PREFILTERED_MASK=pre (or auto|post)
Or unset the variable to fall back to the auto heuristic
Strip whitespace/quotes where the value is injected (CI secrets, Helm values, .env files)

Example fix

# before
export POLARS_PQ_PREFILTERED_MASK=1

# after
export POLARS_PQ_PREFILTERED_MASK=pre
# or: unset POLARS_PQ_PREFILTERED_MASK

Defensive patterns

Strategy: validation

Validate before calling

fn validate_pq_env() -> Result<(), String> {
    if let Ok(v) = std::env::var("POLARS_PQ_PREFILTERED_MASK") {
        if !matches!(v.as_str(), "auto" | "pre" | "post") {
            return Err(format!(
                "POLARS_PQ_PREFILTERED_MASK must be one of auto|pre|post, got {v:?}"
            ));
        }
    }
    Ok(())
}

Try / catch

Deterministic env panic at first predicate scan; catch_unwind adds only a nicer message. Fix or unset the variable.

Prevention

Validate every enum-valued POLARS_* knob against its literal set at startup
Trim whitespace/quotes where CI injects env values
Keep a curated list of sanctioned polars env vars in deployment docs

When it happens

Trigger: export POLARS_PQ_PREFILTERED_MASK=true|1|PRE|"pre " (any non-empty value other than auto/pre/post) followed by any lazy Parquet scan with a filter, e.g. LazyFrame::scan_parquet(path, args)?.filter(col("a").gt(lit(1))).collect().

Common situations: Benchmark tuning copied from blog posts that assume boolean env flags; CI matrices exporting the variable for perf runs; trailing whitespace or quotes introduced by YAML/.env injection.

Understand the failure class

Background: "is not a valid" / "Invalid ... value" environment variable errors: how libraries validate env vars and what to do when they reject yours — this error's family across 48 libraries.

Related errors

AI-assisted analysis of pola-rs/polars@fc24390824 (2026-08-19). Data as JSON: /api/errors/f884e651ae486359. Report an issue: GitHub.