apache/druid · error · IllegalStateException

Task[ ] already exists in taskGroup[ ] with data[ ]

Error message

Task[%s] already exists in taskGroup[%d] with data[%s]

What it means

When the supervisor discovers running tasks (in its task management/run loop), it registers each task into its task group's TaskData map with putIfAbsent. A non-null previous value means the same task id is already registered, implying either a duplicate task launch or state corruption, so an ISE is thrown to surface the inconsistency.

Solutions

  1. Kill the duplicate/stale task via the overlord task API so only one instance remains
  2. Restart the supervisor to rebuild task group state from authoritative task lists
  3. Check for multiple active supervisor leaders or overlapping supervisor restarts; ensure single leadership
  4. Clean orphaned task rows in the metadata storage if re-discovery keeps resurrecting old tasks

Example fix

null
Defensive patterns

Strategy: try-catch

Validate before calling

if (taskGroup.tasks.containsKey(taskId)) {
  log.warn("Task %s already registered in group %s; skipping duplicate registration", taskId, taskGroup.groupId);
  return;
}

Try / catch

try {
  supervisor.registerDiscoveredTask(taskId, taskGroup);
} catch (IllegalStateException e) {
  if (e.getMessage().startsWith("Task[")) {
    // duplicate task; kill the stale instance and restart the supervisor
  } else throw e;
}

Prevention

When it happens

Trigger: Two tasks with identical ids assigned to the same task group — typically caused by duplicate task creation during supervisor restart/race, or the same task being discovered twice by concurrent discovery paths with stale TaskData present.

Common situations: Overlord/task queue duplicate submissions after supervisor failover; leftover task records in metadata storage causing re-discovery of terminated tasks; concurrent supervisor leadership transitions (HA supervisor runners).

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/6aea7f946c6d4114. Report an issue: GitHub.

Appendix: source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java:2584

                                          seekableStreamIndexTask.getIOConfig()
                                                                 .getStartSequenceNumbers()
                                                                 .getPartitionSequenceNumberMap()
                                      ),
                                      null,
                                      endSequences,
                                      seekableStreamIndexTask.getIOConfig().getMinimumMessageTime(),
                                      seekableStreamIndexTask.getIOConfig().getMaximumMessageTime(),
                                      seekableStreamIndexTask.getIOConfig()
                                                             .getStartSequenceNumbers()
                                                             .getExclusivePartitions(),
                                      seekableStreamIndexTask.getIOConfig().getBaseSequenceName()
                                  );
                                }
                            );
                            taskGroupsToVerify.put(taskGroupId, taskGroup);
                            final TaskData prevTaskData = taskGroup.tasks.putIfAbsent(taskId, new TaskData());
                            if (prevTaskData != null) {
                              throw new ISE(
                                  "Task[%s] already exists in taskGroup[%d] with data[%s]",
                                  taskId, taskGroup.groupId, prevTaskData
                              );
                            }
                            final Integer serverPriority = seekableStreamIndexTask.getServerPriority();
                            if (serverPriority != null) {
                              taskGroup.taskIdToServerPriority.putIfAbsent(taskId, serverPriority);
                            }
                            verifySameSequenceNameForAllTasksInGroup(taskGroupId);
                          }
                        }
                        return true;
                      }
                      catch (Throwable t) {
                        stateManager.recordThrowableEvent(t);
                        log.error(t, "An error occurred while discovering task[%s]", taskId);
                        return null;
                      }

View on GitHub (pinned to 9b90983fd2)