{"record":{"id":"a5d98dac13b0a03d","repo":"quickwit-oss/quickwit","slug":"timestamp-secs-must-be-uint64-or-int64-for-mc-3-ch","errorCode":null,"errorMessage":"timestamp_secs must be UInt64 or Int64 for MC-3 check","messagePattern":"timestamp_secs must be UInt64 or Int64 for MC-3 check","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-parquet-engine/src/merge/writer.rs","lineNumber":409,"sourceCode":"        .column(ss_idx)\n        .as_any()\n        .downcast_ref::<BinaryArray>()\n        .expect(\"sorted_series must be Binary\");\n\n    let ts_idx = batch\n        .schema()\n        .index_of(crate::sort_fields::TIMESTAMP_SECS)\n        .expect(\"timestamp_secs column must exist for MC-3 check\");\n    let ts_col = batch.column(ts_idx);\n\n    // Timestamp may be UInt64 or Int64 depending on schema.\n    let ts_values: Vec<i64> =\n        if let Some(arr) = ts_col.as_any().downcast_ref::<arrow::array::UInt64Array>() {\n            arr.values().iter().map(|&v| v as i64).collect()\n        } else if let Some(arr) = ts_col.as_any().downcast_ref::<arrow::array::Int64Array>() {\n            arr.values().to_vec()\n        } else {\n            panic!(\"timestamp_secs must be UInt64 or Int64 for MC-3 check\");\n        };\n\n    for i in 0..batch.num_rows() - 1 {\n        let ss_a = ss_col.value(i);\n        let ss_b = ss_col.value(i + 1);\n\n        match ss_a.cmp(ss_b) {\n            std::cmp::Ordering::Greater => {\n                quickwit_dst::check_invariant!(\n                    quickwit_dst::invariants::InvariantId::MC3,\n                    false,\n                    \": sorted_series decreased at row {}\",\n                    i\n                );\n            }\n            std::cmp::Ordering::Equal => {\n                // Within same series, timestamp must respect the schema direction.\n                // Use the shared compare_with_null_ordering — same function the","sourceCodeStart":391,"sourceCodeEnd":427,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-parquet-engine/src/merge/writer.rs#L391-L427","documentation":"verify_sort_order in the parquet merge writer implements the MC-3 (monotonic-check) validation that timestamp_secs values are non-decreasing across rows. The check only supports timestamp columns materialized as Arrow UInt64Array or Int64Array; any other physical type triggers this panic. It is an internal invariant assertion: the merge planner should only hand it columns whose schema declared a 64-bit timestamp.","triggerScenarios":"Running a merge whose sort key/timestamp column arrives as a different Arrow array type (e.g. TimestampSecond/Milli arrays, Float64, or Dictionary-encoded) instead of plain UInt64/Int64, when called from process_region / process_split_region_col_outer / write_merge_outputs.","commonSituations":"A schema/index-config change altering the stored type of timestamp_secs; a merge pipeline regression passing the wrong column as the sort key; ingestion of documents typed inconsistently with the index mapping.","solutions":["Check the index config: timestamp_secs must be mapped as an i64/u64 (epoch seconds) field, not a fancier timestamp/float type.","Fix the merge writer to cast the timestamp column to Int64Array before verification (arrow::cast) instead of panicking.","Inspect the batch schema at the call site to confirm the column passed as ts_col is actually the timestamp_secs column."],"exampleFix":"// before\n} else {\n    panic!(\"timestamp_secs must be UInt64 or Int64 for MC-3 check\");\n};\n// after\nlet ts_col = arrow::compute::cast(ts_col, &arrow::datatypes::DataType::Int64)\n    .ok_or_else(|| MergeError::Internal(\"timestamp_secs column not castable to Int64\".to_string()))?;\nlet ts_values: Vec<i64> = ts_col\n    .as_any().downcast_ref::<arrow::array::Int64Array>()\n    .expect(\"cast guarantees Int64Array\")\n    .values().to_vec();","handlingStrategy":"validation","validationCode":"use arrow::datatypes::DataType;\nfn ts_col_is_i64_or_u64(schema: &arrow::datatypes::Schema, col: &str) -> bool {\n    matches!(schema.field_with_name(col).unwrap().data_type(),\n        DataType::Int64 | DataType::UInt64)\n}","typeGuard":"fn as_i64_values(col: &dyn arrow::array::Array) -> Option<Vec<i64>> {\n    if let Some(a) = col.as_any().downcast_ref::<arrow::array::Int64Array>() {\n        Some(a.values().to_vec())\n    } else {\n        col.as_any().downcast_ref::<arrow::array::UInt64Array>()\n            .map(|a| a.values().iter().map(|&v| v as i64).collect())\n    }\n}","tryCatchPattern":"// panic is an invariant break; log the batch schema before it fires:\nif as_i64_values(ts_col).is_none() {\n    return Err(anyhow!(\"MC-3: unexpected ts type {:?}\", ts_col.data_type()));\n}","preventionTips":["Assert the batch schema (timestamp_secs is Int64/UInt64) right after record-batch creation in the merge pipeline.","Keep the index config's timestamp field typed as u64/i64 epoch seconds.","Add a unit test feeding a Timestamp-typed column to verify_sort_order to catch regressions."],"tags":["panic","arrow","merge","internal-invariant"],"backgroundTag":"internal-invariant-violation","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}