quickwit-oss/quickwit · error

timestamp_secs must be UInt64 or Int64 for MC-3 check

Error message

timestamp_secs must be UInt64 or Int64 for MC-3 check

What it means

verify_sort_order in the parquet merge writer implements the MC-3 (monotonic-check) validation that timestamp_secs values are non-decreasing across rows. The check only supports timestamp columns materialized as Arrow UInt64Array or Int64Array; any other physical type triggers this panic. It is an internal invariant assertion: the merge planner should only hand it columns whose schema declared a 64-bit timestamp.

Source

Thrown at quickwit/quickwit-parquet-engine/src/merge/writer.rs:409

        .column(ss_idx)
        .as_any()
        .downcast_ref::<BinaryArray>()
        .expect("sorted_series must be Binary");

    let ts_idx = batch
        .schema()
        .index_of(crate::sort_fields::TIMESTAMP_SECS)
        .expect("timestamp_secs column must exist for MC-3 check");
    let ts_col = batch.column(ts_idx);

    // Timestamp may be UInt64 or Int64 depending on schema.
    let ts_values: Vec<i64> =
        if let Some(arr) = ts_col.as_any().downcast_ref::<arrow::array::UInt64Array>() {
            arr.values().iter().map(|&v| v as i64).collect()
        } else if let Some(arr) = ts_col.as_any().downcast_ref::<arrow::array::Int64Array>() {
            arr.values().to_vec()
        } else {
            panic!("timestamp_secs must be UInt64 or Int64 for MC-3 check");
        };

    for i in 0..batch.num_rows() - 1 {
        let ss_a = ss_col.value(i);
        let ss_b = ss_col.value(i + 1);

        match ss_a.cmp(ss_b) {
            std::cmp::Ordering::Greater => {
                quickwit_dst::check_invariant!(
                    quickwit_dst::invariants::InvariantId::MC3,
                    false,
                    ": sorted_series decreased at row {}",
                    i
                );
            }
            std::cmp::Ordering::Equal => {
                // Within same series, timestamp must respect the schema direction.
                // Use the shared compare_with_null_ordering — same function the

View on GitHub (pinned to a39730c5cd)

Solutions

1. Check the index config: timestamp_secs must be mapped as an i64/u64 (epoch seconds) field, not a fancier timestamp/float type.
2. Fix the merge writer to cast the timestamp column to Int64Array before verification (arrow::cast) instead of panicking.
3. Inspect the batch schema at the call site to confirm the column passed as ts_col is actually the timestamp_secs column.

Example fix

// before
} else {
    panic!("timestamp_secs must be UInt64 or Int64 for MC-3 check");
};
// after
let ts_col = arrow::compute::cast(ts_col, &arrow::datatypes::DataType::Int64)
    .ok_or_else(|| MergeError::Internal("timestamp_secs column not castable to Int64".to_string()))?;
let ts_values: Vec<i64> = ts_col
    .as_any().downcast_ref::<arrow::array::Int64Array>()
    .expect("cast guarantees Int64Array")
    .values().to_vec();

use arrow::datatypes::DataType;
fn ts_col_is_i64_or_u64(schema: &arrow::datatypes::Schema, col: &str) -> bool {
    matches!(schema.field_with_name(col).unwrap().data_type(),
        DataType::Int64 | DataType::UInt64)
}

fn as_i64_values(col: &dyn arrow::array::Array) -> Option<Vec<i64>> {
    if let Some(a) = col.as_any().downcast_ref::<arrow::array::Int64Array>() {
        Some(a.values().to_vec())
    } else {
        col.as_any().downcast_ref::<arrow::array::UInt64Array>()
            .map(|a| a.values().iter().map(|&v| v as i64).collect())
    }
}

// panic is an invariant break; log the batch schema before it fires:
if as_i64_values(ts_col).is_none() {
    return Err(anyhow!("MC-3: unexpected ts type {:?}", ts_col.data_type()));
}

When it happens

Trigger: Running a merge whose sort key/timestamp column arrives as a different Arrow array type (e.g. TimestampSecond/Milli arrays, Float64, or Dictionary-encoded) instead of plain UInt64/Int64, when called from process_region / process_split_region_col_outer / write_merge_outputs.

Common situations: A schema/index-config change altering the stored type of timestamp_secs; a merge pipeline regression passing the wrong column as the sort key; ingestion of documents typed inconsistently with the index mapping.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• node not found in pending
• OTP logs or traces do not support VRL transforms
• sort schema '{}' does not contain a timestamp column
• input {} is missing column '{}'
• expected StringArray for service col page

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/a5d98dac13b0a03d. Report an issue: GitHub.