timeseries_id column is required in the batch for sorted_ser

Error message

timeseries_id column is required in the batch for sorted_series key encoding — it is the only guaranteed discriminator for series identity

What it means

resolve_key_columns requires the timeseries_id column to exist in the record batch whenever the sort schema lists it, because it is the only guaranteed discriminator for series identity in the sorted_series key. If the sort schema contains timeseries_id but the batch schema has no such column, this error is raised.

Source

Thrown at quickwit/quickwit-parquet-engine/src/sorted_series/mod.rs:181

/// so the key encoding is consistent: every component gets an ordinal prefix.
///
/// # Errors
///
/// Returns an error if `timeseries_id` is not present in both the sort
/// schema and the batch. It is the only guaranteed discriminator for series
/// identity — without it, different series sharing the same tags would
/// collapse onto the same sorted_series key.
fn resolve_key_columns(
    sort_schema: &quickwit_proto::sortschema::SortSchema,
    batch_schema: &Schema,
) -> Result<ResolvedKeySchema> {
    let mut tag_columns = Vec::new();
    let mut ts_id_column = None;

    for (ordinal, col) in sort_schema.column.iter().enumerate() {
        if col.name == "timeseries_id" {
            let idx = batch_schema.index_of("timeseries_id").map_err(|_| {
                anyhow!(
                    "timeseries_id column is required in the batch for sorted_series key encoding \
                     — it is the only guaranteed discriminator for series identity"
                )
            })?;
            // timeseries_id is a hash — direction is always ascending
            // (it's a tiebreaker, not a semantic ordering).
            ts_id_column = Some(KeyColumn {
                ordinal: ordinal as u8,
                batch_idx: idx,
                descending: false,
            });
            break;
        }
        if crate::sort_fields::is_timestamp_column_name(&col.name) {
            break;
        }
        let is_descending = col.sort_direction
            == quickwit_proto::sortschema::SortColumnDirection::SortDirectionDescending as i32;

View on GitHub (pinned to a39730c5cd)

Solutions

Ensure the batch passed to append_sorted_series_column / compute_sorted_series_column includes a timeseries_id column.
Fix the upstream projection to carry timeseries_id through.
If the column exists under a different name, rename it before calling sorted_series.

Example fix

// before
let batch = batch.project(&other_indices)?; // drops timeseries_id
// after
let mut indices: Vec<usize> = other_indices;
indices.push(batch.schema().index_of("timeseries_id")?);
let batch = batch.project(&indices)?;

Defensive patterns

Strategy: validation

Validate before calling

if sort_schema.column.iter().any(|c| c.name == "timeseries_id")
    && batch.schema().index_of("timeseries_id").is_err()
{
    return Err(anyhow!("batch missing timeseries_id required by sort schema"));
}

Try / catch

match result {
    Err(e) if e.to_string().contains("timeseries_id column is required in the batch") => {
        // recover by re-reading the source with timeseries_id included in the projection
        let batch = reload_with_timeseries_id(source)?;
        // retry
    }
    other => other,
}

Prevention

Never project timeseries_id out of batches destined for sorted_series computation.
Validate batch schema (has timeseries_id) before starting a merge job.
Keep column names stable end-to-end; treat renames as breaking changes.

When it happens

Trigger: compute_sorted_series_column called with a batch missing the timeseries_id column while the configured sort schema includes it — e.g. the batch was projected/renamed upstream or produced by a pipeline step that dropped the column.

Common situations: An upstream projection/select removed timeseries_id; a column rename; feeding exported/subsetted parquet into the streaming merge without the hash column.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/fdf813f7a80e804f. Report an issue: GitHub.