What causes "Failed to publish segments" in apache/druid?

Check overlord and task logs for the underlying publish failure (metadata store errors, timeouts).

apache/druid · critical · IllegalStateException

Failed to publish segments

Error message

Failed to publish segments

What it means

Thrown in the supervisor task's segment-publish path when the publish result indicates segments were not published (the success branch condition failed). After a sequential/single-phase run completes, publishSegments must report success; otherwise the task aborts with this generic ISE.

Source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/common/task/batch/parallel/ParallelIndexSupervisorTask.java:1249

      // segment metrics:
      emitMetric(toolbox.getEmitter(), "ingest/tombstones/count", tombStones.size());
      emitMetric(toolbox.getEmitter(), "ingest/segments/count", newSegments.size());
      emitMetric(toolbox.getEmitter(), "ingest/rows/published", IndexTaskUtils.getTotalRowCount(newSegments));
      // If partitionsSpec is range or hash, we emit info about the size in rows of generated partitions, to detect a hot partition.
      if ((type == SecondaryPartitionType.RANGE || type == SecondaryPartitionType.HASH) && maxRowsPerSegment != null) {
        oversizedSegments = IndexTaskUtils.getOversizedSegments(newSegments, maxRowsPerSegment, DEFAULT_OVERSIZE_RATIO);
        if (oversizedSegments > 0) {
          LOG.warn(
              "Published [%d] oversized segments with more than (maxRowsPerSegment [%d] x ratio [%s]) rows.",
              oversizedSegments,
              maxRowsPerSegment,
              DEFAULT_OVERSIZE_RATIO
          );
          emitMetric(toolbox.getEmitter(), "ingest/segments/oversized", oversizedSegments);
        }
      }
    } else {
      throw new ISE("Failed to publish segments");
    }

    segmentsPublished = (long) newSegments.size();
  }

  private TaskStatus runSequential(TaskToolbox toolbox) throws Exception
  {
    IndexTask sequentialIndexTask = new IndexTask(
        getId(),
        getGroupId(),
        getTaskResource(),
        getDataSource(),
        baseSubtaskSpecName,
        new IndexIngestionSpec(
            getIngestionSchema().getDataSchema(),
            getIngestionSchema().getIOConfig(),
            convertToIndexTuningConfig(getIngestionSchema().getTuningConfig())
        ),

View on GitHub (pinned to 9b90983fd2)

Solutions

Check overlord and task logs for the underlying publish failure (metadata store errors, timeouts).
Verify the metadata store is reachable and has capacity; fix connectivity and rerun the task.
Look for 'published segments' in the task report via the UI to confirm what the publisher returned.
Retry the ingestion task; published segments are idempotent per segment version.

Defensive patterns

Strategy: retry

Try / catch

catch (ISE e) {
  if ("Failed to publish segments".equals(e.getMessage())) {
    // check overlord/metadata-store health, then resubmit the task (publish is idempotent)
  }
}

Prevention

Ensure metadata store (derby/mysql/postgres) is reachable and healthy
Watch overlord logs during publish phase
Avoid overlord restarts mid-task; use HA overlord setup
Retry failed batch tasks — segment publishing is version-idempotent

When it happens

Trigger: Task complete report arrives without a 'published segments' payload (success == false) — e.g. the segment publisher failed, metadata store transaction failed, or the task report is missing the publish section.

Common situations: Metadata store (derby/mysql/postgres) connectivity problems during publish; overlord unable to commit segments; timeouts in segment loading/announce; task reports lost due to overlord restart.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/db0fd275cf75f2d8. Report an issue: GitHub.