input rg col ' ' has no statistics — cannot determine…

Error message

input {input_idx} rg {rg_idx} col '{}' has no statistics — cannot determine prefix alignment without min/max

What it means

extract_rg_composite_prefix_key builds a byte key for each row group from the min/max column statistics; it requires statistics on every prefix column chunk to verify min==max. When parquet::file::metadata returns None statistics for a chunk, the function cannot prove prefix alignment and raises this error instead of guessing.

Solutions

Rewrite/normalize the input parquet files so column chunks carry min/max statistics (re-write with statistics enabled) before the streaming merge.
If the writer is in this repo, ensure statistics are not disabled when writing sorted series inputs.
As a last resort, compute the RG value bounds by decoding the column chunk, but prefer fixing the writer — stats are required by the design.

Example fix

// before: feeding externally-written parquet without stats into streaming merge
// after: normalize input files first, e.g.
let props = WriterProperties::builder().set_statistics_enabled(EnabledStatistics::Page).build();
// and rewrite the file before calling streaming_merge_sorted_parquet_files

Defensive patterns

Strategy: validation

Validate before calling

for rg in 0..meta.num_row_groups() {
    for col_idx in prefix_col_indices {
        if meta.row_group(rg).column(col_idx).statistics().is_none() {
            return Err(anyhow!("input file missing statistics on rg {rg} col {col_idx}"));
        }
    }
}

Try / catch

// inspect and recover
match err.downcast_ref::<String>() {
    Some(msg) if msg.contains("has no statistics") => rewrite_file_with_stats(path)?,
    _ => return Err(err),
}

Prevention

Write all merge-input parquet files with statistics enabled (default writer properties).
Pre-validate inputs with a metadata-only scan before starting a merge job.
Reject externally produced files lacking stats at ingestion time rather than merge time.

When it happens

Trigger: Calling extract_regions_from_metadata (or streaming_merge_sorted_parquet_files) on a parquet file whose row-group column chunks were written without statistics — e.g. written with set_statistics disabled or by a writer that omits stats for that column.

Common situations: Files produced by external/older writers with statistics turned off; columns where the writer intentionally skipped stats; synthetic test files built without stats.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/99b9e9f42195ceb1. Report an issue: GitHub.

Appendix: source

Thrown at quickwit/quickwit-parquet-engine/src/merge/streaming/region_grouping.rs:241

    metadata: &ParquetMetaData,
    rg_idx: usize,
    prefix_cols: &[Option<PrefixColumn>],
    input_idx: usize,
) -> Result<Vec<u8>> {
    let rg_meta = metadata.row_group(rg_idx);
    let mut key = Vec::new();
    for col_opt in prefix_cols {
        let Some(col) = col_opt else {
            // SS-3 implicit null: column absent from schema, so every
            // row's value is null. Skip the slot entirely — the
            // trailing prefix-length sentinel will keep this from
            // colliding with present-value keys, and sorted_series
            // applies the same "skip null cols" rule at the row level.
            continue;
        };
        let chunk = rg_meta.column(col.parquet_col_idx);
        let stats = chunk.statistics().ok_or_else(|| {
            anyhow!(
                "input {input_idx} rg {rg_idx} col '{}' has no statistics — cannot determine \
                 prefix alignment without min/max",
                col.name,
            )
        })?;

        // Parquet's `num_values` is total cell count including nulls.
        // `null_count_opt()` returns the explicitly-recorded null
        // count (defaulting to 0 when absent, per parquet-rs guidance).
        let num_values = chunk.num_values().max(0) as u64;
        let null_count = stats.null_count_opt().unwrap_or(0);

        if num_values > 0 && null_count == num_values {
            // All-null RG: skip the column entirely (don't write its
            // ordinal byte or value). The trailing prefix-length
            // sentinel below ensures the resulting all-null key
            // still sorts after any non-null key. See the sentinel
            // comment for the full argument.

View on GitHub (pinned to a39730c5cd)