{"record":{"id":"fdf813f7a80e804f","repo":"quickwit-oss/quickwit","slug":"timeseries-id-column-is-required-in-the-batch-for","errorCode":null,"errorMessage":"timeseries_id column is required in the batch for sorted_series key encoding — it is the only guaranteed discriminator for series identity","messagePattern":"timeseries_id column is required in the batch for sorted_series key encoding — it is the only guaranteed discriminator for series identity","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-parquet-engine/src/sorted_series/mod.rs","lineNumber":181,"sourceCode":"/// so the key encoding is consistent: every component gets an ordinal prefix.\n///\n/// # Errors\n///\n/// Returns an error if `timeseries_id` is not present in both the sort\n/// schema and the batch. It is the only guaranteed discriminator for series\n/// identity — without it, different series sharing the same tags would\n/// collapse onto the same sorted_series key.\nfn resolve_key_columns(\n    sort_schema: &quickwit_proto::sortschema::SortSchema,\n    batch_schema: &Schema,\n) -> Result<ResolvedKeySchema> {\n    let mut tag_columns = Vec::new();\n    let mut ts_id_column = None;\n\n    for (ordinal, col) in sort_schema.column.iter().enumerate() {\n        if col.name == \"timeseries_id\" {\n            let idx = batch_schema.index_of(\"timeseries_id\").map_err(|_| {\n                anyhow!(\n                    \"timeseries_id column is required in the batch for sorted_series key encoding \\\n                     — it is the only guaranteed discriminator for series identity\"\n                )\n            })?;\n            // timeseries_id is a hash — direction is always ascending\n            // (it's a tiebreaker, not a semantic ordering).\n            ts_id_column = Some(KeyColumn {\n                ordinal: ordinal as u8,\n                batch_idx: idx,\n                descending: false,\n            });\n            break;\n        }\n        if crate::sort_fields::is_timestamp_column_name(&col.name) {\n            break;\n        }\n        let is_descending = col.sort_direction\n            == quickwit_proto::sortschema::SortColumnDirection::SortDirectionDescending as i32;","sourceCodeStart":163,"sourceCodeEnd":199,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-parquet-engine/src/sorted_series/mod.rs#L163-L199","documentation":"resolve_key_columns requires the timeseries_id column to exist in the record batch whenever the sort schema lists it, because it is the only guaranteed discriminator for series identity in the sorted_series key. If the sort schema contains timeseries_id but the batch schema has no such column, this error is raised.","triggerScenarios":"compute_sorted_series_column called with a batch missing the timeseries_id column while the configured sort schema includes it — e.g. the batch was projected/renamed upstream or produced by a pipeline step that dropped the column.","commonSituations":"An upstream projection/select removed timeseries_id; a column rename; feeding exported/subsetted parquet into the streaming merge without the hash column.","solutions":["Ensure the batch passed to append_sorted_series_column / compute_sorted_series_column includes a timeseries_id column.","Fix the upstream projection to carry timeseries_id through.","If the column exists under a different name, rename it before calling sorted_series."],"exampleFix":"// before\nlet batch = batch.project(&other_indices)?; // drops timeseries_id\n// after\nlet mut indices: Vec<usize> = other_indices;\nindices.push(batch.schema().index_of(\"timeseries_id\")?);\nlet batch = batch.project(&indices)?;","handlingStrategy":"validation","validationCode":"if sort_schema.column.iter().any(|c| c.name == \"timeseries_id\")\n    && batch.schema().index_of(\"timeseries_id\").is_err()\n{\n    return Err(anyhow!(\"batch missing timeseries_id required by sort schema\"));\n}\n","typeGuard":null,"tryCatchPattern":"match result {\n    Err(e) if e.to_string().contains(\"timeseries_id column is required in the batch\") => {\n        // recover by re-reading the source with timeseries_id included in the projection\n        let batch = reload_with_timeseries_id(source)?;\n        // retry\n    }\n    other => other,\n}\n","preventionTips":["Never project timeseries_id out of batches destined for sorted_series computation.","Validate batch schema (has timeseries_id) before starting a merge job.","Keep column names stable end-to-end; treat renames as breaking changes."],"tags":["sorted-series","schema","missing-column","merge"],"backgroundTag":"missing-required-argument","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}