{"record":{"id":"54e61012f23490ec","repo":"quickwit-oss/quickwit","slug":"sort-schema-does-not-contain-a-timestamp-colu","errorCode":null,"errorMessage":"sort schema '{}' does not contain a timestamp column","messagePattern":"sort schema '(.+?)' does not contain a timestamp column","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-parquet-engine/src/merge/merge_order.rs","lineNumber":137,"sourceCode":"/// position in the output, depending on sort order.\n///\n/// Returns an RLE-encoded merge order: contiguous runs from the same input\n/// are collapsed into a single `MergeRun`.\npub fn compute_merge_order(inputs: &[RecordBatch], sort_fields_str: &str) -> Result<Vec<MergeRun>> {\n    if inputs.is_empty() {\n        return Ok(Vec::new());\n    }\n\n    // Parse the sort schema to determine timestamp sort direction.\n    // Legacy schemas may use \"timestamp\" instead of \"timestamp_secs\".\n    let sort_schema = parse_sort_fields(sort_fields_str)?;\n\n    let ts_column = sort_schema\n        .column\n        .iter()\n        .find(|c| is_timestamp_column_name(&c.name))\n        .ok_or_else(|| {\n            anyhow::anyhow!(\n                \"sort schema '{}' does not contain a timestamp column\",\n                sort_fields_str,\n            )\n        })?;\n\n    let ts_descending = ts_column.sort_direction\n        == quickwit_proto::sortschema::SortColumnDirection::SortDirectionDescending as i32;\n\n    // Determine the timestamp column data type from the first non-empty input.\n    let ts_data_type = inputs\n        .iter()\n        .find(|b| b.num_rows() > 0)\n        .map(|b| {\n            let schema = b.schema();\n            let (_, field) = schema\n                .column_with_name(TIMESTAMP_SECS)\n                .expect(\"timestamp_secs column must exist\");\n            field.data_type().clone()","sourceCodeStart":119,"sourceCodeEnd":155,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-parquet-engine/src/merge/merge_order.rs#L119-L155","documentation":"compute_merge_order determines the merge order from the sort schema embedded in the parquet files by locating the timestamp column via is_timestamp_column_name. If the schema's sort columns contain no recognized timestamp column name, the merge order cannot be computed and the function fails, including the sort schema string for diagnosis.","triggerScenarios":"Merging parquet splits whose PARQUET_META_SORT_FIELDS metadata points to sort columns that don't include a timestamp-named column (e.g. sorted only by service name or by an arbitrary field).","commonSituations":"Files written by a different producer/version that sorts by non-timestamp keys; misconfigured indexer emitting sort fields without the timestamp column; hand-crafted parquet inputs fed to the merge API.","solutions":["Ensure inputs are sorted with the timestamp column included in the sort fields before merging.","Check the PARQUET_META_SORT_FIELDS key-value metadata of the offending file and align it with Quickwit's expected timestamp column naming.","Re-write the input files with the correct sort order, or exclude them from the merge set."],"exampleFix":"// before: sort fields without timestamp\nSortField { name: \"service\" , desc: true }\n// after: include the timestamp column\nSortField { name: \"_timestamp\", desc: true }, SortField { name: \"service\", desc: true }","handlingStrategy":"validation","validationCode":"// before merging, check the sort fields metadata of each input\nlet sort_fields = read_parquet_kv_metadata(path, PARQUET_META_SORT_FIELDS)?;\nassert!(sort_fields.iter().any(|f| is_timestamp_column_name(&f.name)),\n        \"{} has no timestamp sort column\", path.display());","typeGuard":null,"tryCatchPattern":"match compute_merge_order(&inputs) {\n    Err(e) if e.to_string().contains(\"does not contain a timestamp column\") => {\n        // re-sort the input files with the timestamp column or skip them\n    }\n    other => other?,\n}","preventionTips":["Always include the timestamp column in sort fields when writing split files.","Only feed Quickwit-written parquet files into the merge pipeline.","Validate PARQUET_META_SORT_FIELDS of all inputs before merging."],"tags":["parquet","arrow","merge","schema"],"backgroundTag":"schema-validation-failed","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}