apache/druid · error · IllegalStateException

Failed to publish schemas

Error message

Failed to publish schemas[%s] to DB for datasource[%s] and version[%s]

What it means

SegmentSchemaManager.publishSchemasToDB inserts segment schema records in batches into the metadata store; any row whose insert failed is collected in failedInserts. If any insert in a partition fails, this IllegalStateException is thrown to abort persistSchemaAndUpdateSegmentsTable, since publishing schema metadata partially would leave the segments table inconsistent with the schema table.

Solutions

1. Check metadata-store connectivity and logs from the metadata storage connector for the root SQL error immediately preceding this ISE
2. Retry persistSchemaAndUpdateSegmentsTable — inserts are batched and the operation is designed to be retried after transient DB failures
3. Inspect the failedInserts list in the message to identify which schemas failed and check for constraint/key conflicts on those segment IDs
4. Verify druid.metadata.storage.connector configuration and DB schema migrations are current for the schema tables

Example fix

// before
segmentSchemaManager.persistSchemaAndUpdateSegmentsTable(segments);
// after
try {
  segmentSchemaManager.persistSchemaAndUpdateSegmentsTable(segments);
} catch (ISE e) {
  log.warn(e, "Schema publish failed; will retry after metadata store recovers");
  throw e;
}

// Verify metadata store reachability before publishing
try (MetadataStorageConnector conn = connectorFactory.getConnector()) {
  conn.lookup(CONFIG_TABLE_KEY); // throws if DB is unreachable
}

try {
  manager.persistSchemaAndUpdateSegmentsTable(segments);
} catch (ISE e) {
  log.warn(e, "Transient schema publish failure; scheduling retry");
  scheduleRetry();
}

When it happens

Trigger: Calling persistSchemaAndUpdateSegmentsTable (typically from indexing-task cleanup/publish paths) when a batch insert into druid_segment_schema via the metadata storage connector fails for one or more rows — e.g. metadata DB outage, connection pool exhaustion, constraint violation, or oversized schema payload.

Common situations: Metadata database unreachable or failing over during segment publish; schema rows too large for the configured column type; duplicate/unique-key conflicts after task retries; network blips between Druid and the metadata store.

Related errors

• Failed to update segments with schema information
• Attempt to add row to swapped-out sink for segment
• authResult.getErrorMessage()
• Bloom filter aggregators are query-time only
• Can't find previous segmentIds for sequence

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/2e3a107cc71051bd. Report an issue: GitHub.

Appendix: source

Thrown at server/src/main/java/org/apache/druid/segment/metadata/SegmentSchemaManager.java:272

                         .bind("payload", jsonMapper.writeValueAsBytes(fingerprintSchemaPayloadMap.get(fingerprint)))
                         .bind("used", true)
                         .bind("used_status_last_updated", now)
                         .bind("version", version);
      }
      final int[] affectedRows = schemaInsertBatch.execute();
      final List<String> failedInserts = new ArrayList<>();
      for (int i = 0; i < partition.size(); ++i) {
        if (affectedRows[i] != 1) {
          failedInserts.add(partition.get(i));
        }
      }
      if (failedInserts.isEmpty()) {
        log.info(
            "Published schemas [%s] to DB for datasource[%s] and version[%s].",
            partition, dataSource, version
        );
      } else {
        throw new ISE(
            "Failed to publish schemas[%s] to DB for datasource[%s] and version[%s]",
            failedInserts, dataSource, version
        );
      }
    }
  }

  /**
   * Update segment with schemaFingerprint and numRows information.
   */
  public void updateSegmentWithSchemaInformation(
      final Handle handle,
      final List<SegmentSchemaMetadataPlus> batch,
      final DateTime updateTime
  )
  {
    log.debug("Updating segment with schemaFingerprint and numRows information: [%s].", batch);

View on GitHub (pinned to 9b90983fd2)