nautechsystems/nautilus_trader · error · anyhow::Error

Warning: Statistics not available for column '{column_name}'

Error message

Warning: Statistics not available for column '{column_name}' in row group {i}.

What it means

min_max_from_parquet_metadata_object_store needs per-row-group min/max statistics for the timestamp column to compute file bounds. If a row group lacks statistics for that column, it bails with this message, because the file's true min/max cannot be determined from metadata alone.

Source

Thrown at crates/persistence/src/parquet.rs:453

                    if let Statistics::Int64(int64_stats) = stats {
                        // Extract min value if available
                        if let Some(&min_value) = int64_stats.min_opt()
                            && overall_min_value.is_none_or(|overall_min| min_value < overall_min)
                        {
                            overall_min_value = Some(min_value);
                        }

                        // Extract max value if available
                        if let Some(&max_value) = int64_stats.max_opt()
                            && overall_max_value.is_none_or(|overall_max| max_value > overall_max)
                        {
                            overall_max_value = Some(max_value);
                        }
                    } else {
                        anyhow::bail!("Warning: Column name '{column_name}' is not of type i64.");
                    }
                } else {
                    anyhow::bail!(
                        "Warning: Statistics not available for column '{column_name}' in row group {i}."
                    );
                }
            }
        }
    }

    // Return the min/max pair if both are available
    if let (Some(min), Some(max)) = (overall_min_value, overall_max_value) {
        Ok((
            u64::try_from(min).map_err(|_| {
                anyhow::anyhow!("Negative minimum value {min} for column '{column_name}'")
            })?,
            u64::try_from(max).map_err(|_| {
                anyhow::anyhow!("Negative maximum value {max} for column '{column_name}'")
            })?,
        ))
    } else {

View on GitHub (pinned to 18893faf8b)

Solutions

  1. Rewrite the file with column statistics enabled (nautilus' writer enables them by default).
  2. Verify the column name matches the actual timestamp column in the schema.
  3. Recompute the file bounds by reading the data instead of metadata, then rebuild/reset names accordingly.
  4. Re-export the data from the original writer with default statistics settings.

Example fix

// before — writing without stats
writer_properties.set_statistics_enabled(false);
// after
let props = WriterProperties::builder().build(); // stats enabled by default
Defensive patterns

Strategy: validation

Validate before calling

let meta = reader.metadata()?;
let rg0 = meta.row_group(0);
let has_stats = rg0.column(0).statistics().is_some();
if !has_stats {
    eprintln!("file written without column statistics; rewrite before reset_file_names");
}

Try / catch

match min_max_from_parquet_metadata(meta, col) {
    Err(e) if e.to_string().contains("Statistics not available") => {
        // rewrite the file with statistics enabled, then retry
    }
    other => other?,
}

Prevention

When it happens

Trigger: Reading a Parquet file written with statistics disabled (set_statistics_enabled(false)), files written by external tools without column statistics, or a column name that doesn't exist in the row group schema.

Common situations: Files produced by pyarrow with stats disabled for size/speed; catalogs repacked by third-party tools; passing a wrong column name so statistics look 'unavailable'.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/3be6d6b71391b284. Report an issue: GitHub.