Topic/stream in metadata storage

Error message

Topic/stream in metadata storage [%s] doesn't match spec topic/stream [%s], ignoring stored sequences

What it means

In KafkaSupervisor.getOffsetsFromMetadataStorage(), sequences stored in metadata storage belong to a topic/stream that does not match the current spec's topic. The supervisor logs a warning (once per mismatched topic) and ignores those stored sequence numbers, effectively starting fresh for the spec's topic. This guards against reusing offsets from a different topic.

Solutions

Update the supervisor spec so the topic matches what metadata storage holds, or accept the reset.
If a topic change is intended, also reset the datasource (reset supervisor) and expect consumption from scratch or earliest.
Clean stale supervisor entries in the metadata storage table if they belong to deleted specs.
Use a unique datasource name for a new topic to avoid colliding stored offsets.

Example fix

// before (spec edited in place)
"topic": "new-topic" // metadata store still has old-topic offsets -> mismatch warning
// after
POST /druid/indexer/supervisor/<id>/reset  // clear stale offsets for the datasource
then submit spec with "topic": "new-topic"

Defensive patterns

Strategy: validation

Validate before calling

// Compare spec topic with stored supervisor state before submitting
SupervisorMetaData md = fetchMetaDataStoreEntry(datasource);
if (md != null && !Objects.equals(md.getSpec().getIoConfig().getTopic(), newSpec.getIoConfig().getTopic())) { resetSupervisor(datasource); }

Prevention

Reset the supervisor when intentionally changing the topic
Use a distinct datasource name per topic
Clean up metadata-store rows of retired supervisors

When it happens

Trigger: The spec's topic/stream was changed (or metadata storage holds entries from a previous spec) so getMatchingKafkaTopicPartition returns null for the stored stream name.

Common situations: Editing an existing supervisor spec to point at a new topic while reusing the old datasource name; copied/renamed supervisor configs; stale metadata-store rows from an earlier spec version.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/0b7e3fa96e02bc5f. Report an issue: GitHub.

Appendix: source

Thrown at extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/supervisor/KafkaSupervisor.java:667

      SeekableStreamSequenceNumbers<KafkaTopicPartition, Long> partitions = ((KafkaDataSourceMetadata) dataSourceMetadata)
          .getSeekableStreamSequenceNumbers();
      if (partitions != null && partitions.getPartitionSequenceNumberMap() != null) {
        Map<KafkaTopicPartition, Long> partitionOffsets = new HashMap<>();
        Set<String> topicMisMatchLogged = new HashSet<>();
        partitions.getPartitionSequenceNumberMap().forEach((kafkaTopicPartition, value) -> {
          final String matchValue;
          // previous offsets are from multi-topic config
          if (kafkaTopicPartition.topic().isPresent()) {
            matchValue = kafkaTopicPartition.topic().get();
          } else {
            // previous offsets are from single topic config
            matchValue = partitions.getStream();
          }

          KafkaTopicPartition matchingTopicPartition = getMatchingKafkaTopicPartition(kafkaTopicPartition, matchValue);

          if (matchingTopicPartition == null && !topicMisMatchLogged.contains(matchValue)) {
            log.warn(
                "Topic/stream in metadata storage [%s] doesn't match spec topic/stream [%s], ignoring stored sequences",
                matchValue,
                getIoConfig().getStream()
            );
            topicMisMatchLogged.add(matchValue);
          }
          if (matchingTopicPartition != null) {
            partitionOffsets.put(matchingTopicPartition, value);
          }
        });
        return partitionOffsets;
      }
    }

    return Collections.emptyMap();
  }

  @Nullable

View on GitHub (pinned to 9b90983fd2)