{"record":{"id":"a1990a2b85274f2c","repo":"apache/druid","slug":"transaction-failure-publishing-segments-for-sequen","errorCode":null,"errorMessage":"Transaction failure publishing segments for sequence [%s]","messagePattern":"Transaction failure publishing segments for sequence \\[(.+?)\\]","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/SeekableStreamIndexTaskRunner.java","lineNumber":1073,"sourceCode":"\n  private void publishAndRegisterHandoff(SequenceMetadata<PartitionIdType, SequenceOffsetType> sequenceMetadata)\n  {\n    log.debug(\"Publishing segments for sequence [%s].\", sequenceMetadata);\n\n    // annotateSegmentWithPartitionDimensionValues returns the segment unchanged when there is no shardSpecCollector,\n    // so it is always safe to apply here.\n    final ListenableFuture<SegmentsAndCommitMetadata> publishFuture = Futures.transform(\n        driver.publish(\n            sequenceMetadata.createPublisher(this, toolbox, ioConfig.isUseTransaction()),\n            sequenceMetadata.getCommitterSupplier(this, stream, lastPersistedOffsets).get(),\n            Collections.singletonList(sequenceMetadata.getSequenceName()),\n            segments -> segments.stream()\n                                .map(this::annotateSegmentWithPartitionDimensionValues)\n                                .collect(Collectors.toCollection(LinkedHashSet::new))\n        ),\n        publishedSegmentsAndMetadata -> {\n          if (publishedSegmentsAndMetadata == null) {\n            throw new ISE(\n                \"Transaction failure publishing segments for sequence [%s]\",\n                sequenceMetadata\n            );\n          } else {\n            return publishedSegmentsAndMetadata;\n          }\n        },\n        MoreExecutors.directExecutor()\n    );\n    publishWaitList.add(publishFuture);\n\n    // Create a handoffFuture for every publishFuture. The created handoffFuture must fail if publishFuture fails.\n    final SettableFuture<SegmentsAndCommitMetadata> handoffFuture = SettableFuture.create();\n    handOffWaitList.add(handoffFuture);\n\n    Futures.addCallback(\n        publishFuture,\n        new FutureCallback<>()","sourceCodeStart":1055,"sourceCodeEnd":1091,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/SeekableStreamIndexTaskRunner.java#L1055-L1091","documentation":"In SeekableStreamIndexTaskRunner, after reading all data for a sequence, the task calls taskActionClient.submit(TransactionAppendedAndPublishAction)-style publish inside a transaction that appends and publishes segments atomically. The transaction callback receives null when the transaction fails to commit (e.g. coordinator/task action failure, lock revocation, or segment publish conflict), and the runner converts that into an IllegalStateException 'Transaction failure publishing segments for sequence [%s]'.","triggerScenarios":"The indexing-task publish transaction (initTransaction/publish) returns null because the task action call to the overlord failed, the task's locks were revoked, a segment conflict/overwrite occurred, or the coordinator rejected the commit for sequence %s.","commonSituations":"Coordinator restart or unresponsiveness during segment publish; task lock contention with a concurrent compaction or manual kill; ZK/overlord connectivity problems; metadata store (metadata DB) outage during commit.","solutions":["Inspect overlord/coordinator and task logs for the underlying task-action failure (lock revoked, action timed out, DB error) and fix that root cause.","Resume/restart the task: Druid tasks are restartable from their checkpoints; the sequence will be republished.","Ensure no competing supervisor/compaction is writing overlapping intervals while the task publishes; stagger compaction.","Verify metadata storage health and connectivity from overlord and middle managers."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { publishSegments(sequence); } catch (ISE e) { // transaction returned null; task is restartable\n resumeTaskViaSupervisor(taskId); }","preventionTips":["Avoid overlapping compaction/kill jobs while streaming tasks publish","Monitor overlord/coordinator and metadata DB health","Rely on Druid's task restart-from-checkpoint semantics"],"tags":["druid","streaming","transaction","segment-publish"],"backgroundTag":"database-write-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}