apache/druid · error · IllegalStateException

Transaction failure publishing segments for sequence [%s]

Error message

Transaction failure publishing segments for sequence [%s]

What it means

In SeekableStreamIndexTaskRunner, after reading all data for a sequence, the task calls taskActionClient.submit(TransactionAppendedAndPublishAction)-style publish inside a transaction that appends and publishes segments atomically. The transaction callback receives null when the transaction fails to commit (e.g. coordinator/task action failure, lock revocation, or segment publish conflict), and the runner converts that into an IllegalStateException 'Transaction failure publishing segments for sequence [%s]'.

Source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/SeekableStreamIndexTaskRunner.java:1073

  private void publishAndRegisterHandoff(SequenceMetadata<PartitionIdType, SequenceOffsetType> sequenceMetadata)
  {
    log.debug("Publishing segments for sequence [%s].", sequenceMetadata);

    // annotateSegmentWithPartitionDimensionValues returns the segment unchanged when there is no shardSpecCollector,
    // so it is always safe to apply here.
    final ListenableFuture<SegmentsAndCommitMetadata> publishFuture = Futures.transform(
        driver.publish(
            sequenceMetadata.createPublisher(this, toolbox, ioConfig.isUseTransaction()),
            sequenceMetadata.getCommitterSupplier(this, stream, lastPersistedOffsets).get(),
            Collections.singletonList(sequenceMetadata.getSequenceName()),
            segments -> segments.stream()
                                .map(this::annotateSegmentWithPartitionDimensionValues)
                                .collect(Collectors.toCollection(LinkedHashSet::new))
        ),
        publishedSegmentsAndMetadata -> {
          if (publishedSegmentsAndMetadata == null) {
            throw new ISE(
                "Transaction failure publishing segments for sequence [%s]",
                sequenceMetadata
            );
          } else {
            return publishedSegmentsAndMetadata;
          }
        },
        MoreExecutors.directExecutor()
    );
    publishWaitList.add(publishFuture);

    // Create a handoffFuture for every publishFuture. The created handoffFuture must fail if publishFuture fails.
    final SettableFuture<SegmentsAndCommitMetadata> handoffFuture = SettableFuture.create();
    handOffWaitList.add(handoffFuture);

    Futures.addCallback(
        publishFuture,
        new FutureCallback<>()

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Inspect overlord/coordinator and task logs for the underlying task-action failure (lock revoked, action timed out, DB error) and fix that root cause.
2. Resume/restart the task: Druid tasks are restartable from their checkpoints; the sequence will be republished.
3. Ensure no competing supervisor/compaction is writing overlapping intervals while the task publishes; stagger compaction.
4. Verify metadata storage health and connectivity from overlord and middle managers.

try { publishSegments(sequence); } catch (ISE e) { // transaction returned null; task is restartable
 resumeTaskViaSupervisor(taskId); }

When it happens

Trigger: The indexing-task publish transaction (initTransaction/publish) returns null because the task action call to the overlord failed, the task's locks were revoked, a segment conflict/overwrite occurred, or the coordinator rejected the commit for sequence %s.

Common situations: Coordinator restart or unresponsiveness during segment publish; task lock contention with a concurrent compaction or manual kill; ZK/overlord connectivity problems; metadata store (metadata DB) outage during commit.

Related errors

• Root partition range[%d, %d] of new segments doesn't match t
• New segment[%s] have a smaller minor version than old segmen
• Stream ingestion task unexpectedly attempted to overwrite se
• Failed to load next batch of results
• Emit called unexpectedly before service start

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/a1990a2b85274f2c. Report an issue: GitHub.