risingwavelabs/risingwave · error · BatchError

cannot create a duplicate task with the same ID: {:?}

Error message

cannot create a duplicate task with the same ID: {:?}

What it means

A duplicate-key validation guard in TaskManager::fire_task: before spawning a batch task it inserts into the tasks map, and if a TaskId equal to the incoming one is already present the insert fails with this error. TaskIds embed query/stage/epoch identifiers and should be unique per (query, stage, epoch), so a collision means the same task was fired twice — a scheduler retry bug, a duplicate gRPC CreateTask, or an epoch reuse in the meta service.

Source

Thrown at src/batch/src/task/task_manager.rs:144

        let task_id = task.get_task_id().clone();
        let task = Arc::new(task);
        // Here the task id insert into self.tasks is put in front of `.async_execute`, cuz when
        // send `TaskStatus::Running` in `.async_execute`, the query runner may schedule next stage,
        // it's possible do not found parent task id in theory.
        let ret = if let hash_map::Entry::Vacant(e) = self.tasks.lock().entry(task_id.clone()) {
            e.insert(task.clone());

            let this = self.clone();
            let task_id = task_id.clone();
            let state_reporter = state_reporter.clone();
            let heartbeat_join_handle = self.runtime.spawn(async move {
                this.start_task_heartbeat(state_reporter, task_id).await;
            });
            task.set_heartbeat_join_handle(heartbeat_join_handle);

            Ok(())
        } else {
            bail!(
                "cannot create a duplicate task with the same ID: {:?}",
                task_id,
            );
        };
        task.async_execute(Some(state_reporter), tracing_context, expr_context)
            .await
            .inspect_err(|_| {
                self.cancel_task(&task_id.to_prost());
            })?;
        ret
    }

    #[cfg(test)]
    async fn fire_task_for_test(
        self: &Arc<Self>,
        tid: &PbTaskId,
        plan: PlanFragment,
    ) -> Result<()> {

View on GitHub (pinned to 6469eb736d)

Solutions

1. Make task creation idempotent: if the existing task has the same identity, return the already-running task instead of erroring
2. Ensure the scheduler does not re-send CreateTask for the same (query_id, stage_id, epoch) after a timeout without first checking task status
3. Check meta-node epoch allocation if epochs are being reused across retries
4. Log both the existing and incoming task contexts to diagnose whether the duplicate came from a retry or a client bug

When it happens

Trigger: Thrown at src/batch/src/task/task_manager.rs:144 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/6b97b57089227b3e. Report an issue: GitHub.