What causes "sort schema '{}' does not contain a timestamp column" in quickwit-oss/quickwit?

Ensure inputs are sorted with the timestamp column included in the sort fields before merging.

sort schema '{}' does not contain a timestamp column

Error message

sort schema '{}' does not contain a timestamp column

What it means

compute_merge_order determines the merge order from the sort schema embedded in the parquet files by locating the timestamp column via is_timestamp_column_name. If the schema's sort columns contain no recognized timestamp column name, the merge order cannot be computed and the function fails, including the sort schema string for diagnosis.

Source

Thrown at quickwit/quickwit-parquet-engine/src/merge/merge_order.rs:137

/// position in the output, depending on sort order.
///
/// Returns an RLE-encoded merge order: contiguous runs from the same input
/// are collapsed into a single `MergeRun`.
pub fn compute_merge_order(inputs: &[RecordBatch], sort_fields_str: &str) -> Result<Vec<MergeRun>> {
    if inputs.is_empty() {
        return Ok(Vec::new());
    }

    // Parse the sort schema to determine timestamp sort direction.
    // Legacy schemas may use "timestamp" instead of "timestamp_secs".
    let sort_schema = parse_sort_fields(sort_fields_str)?;

    let ts_column = sort_schema
        .column
        .iter()
        .find(|c| is_timestamp_column_name(&c.name))
        .ok_or_else(|| {
            anyhow::anyhow!(
                "sort schema '{}' does not contain a timestamp column",
                sort_fields_str,
            )
        })?;

    let ts_descending = ts_column.sort_direction
        == quickwit_proto::sortschema::SortColumnDirection::SortDirectionDescending as i32;

    // Determine the timestamp column data type from the first non-empty input.
    let ts_data_type = inputs
        .iter()
        .find(|b| b.num_rows() > 0)
        .map(|b| {
            let schema = b.schema();
            let (_, field) = schema
                .column_with_name(TIMESTAMP_SECS)
                .expect("timestamp_secs column must exist");
            field.data_type().clone()

View on GitHub (pinned to a39730c5cd)

Solutions

Ensure inputs are sorted with the timestamp column included in the sort fields before merging.
Check the PARQUET_META_SORT_FIELDS key-value metadata of the offending file and align it with Quickwit's expected timestamp column naming.
Re-write the input files with the correct sort order, or exclude them from the merge set.

Example fix

// before: sort fields without timestamp
SortField { name: "service" , desc: true }
// after: include the timestamp column
SortField { name: "_timestamp", desc: true }, SortField { name: "service", desc: true }

Defensive patterns

Strategy: validation

Validate before calling

// before merging, check the sort fields metadata of each input
let sort_fields = read_parquet_kv_metadata(path, PARQUET_META_SORT_FIELDS)?;
assert!(sort_fields.iter().any(|f| is_timestamp_column_name(&f.name)),
        "{} has no timestamp sort column", path.display());

Try / catch

match compute_merge_order(&inputs) {
    Err(e) if e.to_string().contains("does not contain a timestamp column") => {
        // re-sort the input files with the timestamp column or skip them
    }
    other => other?,
}

Prevention

Always include the timestamp column in sort fields when writing split files.
Only feed Quickwit-written parquet files into the merge pipeline.
Validate PARQUET_META_SORT_FIELDS of all inputs before merging.

When it happens

Trigger: Merging parquet splits whose PARQUET_META_SORT_FIELDS metadata points to sort columns that don't include a timestamp-named column (e.g. sorted only by service name or by an arbitrary field).

Common situations: Files written by a different producer/version that sorts by non-timestamp keys; misconfigured indexer emitting sort fields without the timestamp column; hand-crafted parquet inputs fed to the merge API.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/54e61012f23490ec. Report an issue: GitHub.