What causes "Already shut down, not starting again" in apache/druid?

Create a new KafkaLookupExtractorFactory instance instead of reusing a stopped one.

Already shut down, not starting again

Error message

Already shut down, not starting again

What it means

KafkaLookupExtractorFactory.start() returns false instead of starting the Kafka consumer when the executor service backing the lookup has already been shut down. The factory keeps an internal 'started' flag and executor; once stop()/shutdown has run, a restart is deliberately refused so lookups never half-start against dead resources. It is a returned boolean (logged at warn), not an exception.

Source

Thrown at extensions-core/kafka-extraction-namespace/src/main/java/org/apache/druid/query/lookup/KafkaLookupExtractorFactory.java:161

  {
    return connectTimeout;
  }

  public boolean isInjective()
  {
    return injective;
  }

  @Override
  public boolean start()
  {
    synchronized (started) {
      if (started.get()) {
        LOG.warn("Already started, not starting again");
        return true;
      }
      if (executorService.isShutdown()) {
        LOG.warn("Already shut down, not starting again");
        return false;
      }
      verifyKafkaProperties();

      final String topic = getKafkaTopic();
      LOG.debug("About to listen to topic [%s] with group.id [%s]", topic, factoryId);
      // this creates a ConcurrentMap
      cacheHandler = cacheManager.createCache();
      final Map<String, String> map = cacheHandler.getCache();
      mapRef.set(map);


      final CountDownLatch startingReads = new CountDownLatch(1);

      final ListenableFuture<?> future = executorService.submit(() -> {
        final Consumer<String, String> consumer = getConsumer();
        consumer.subscribe(Collections.singletonList(topic));
        try {

View on GitHub (pinned to 9b90983fd2)

Solutions

Create a new KafkaLookupExtractorFactory instance instead of reusing a stopped one.
Check the return value of start() and rebuild the factory if it returns false.
Do not call stop() on the factory unless the lookup is permanently retired.
Verify lifecycle ordering in the lookup extraction manager so start() is never invoked after shutdown.

Example fix

// before
factory.stop();
boolean ok = factory.start(); // returns false: "Already shut down"
// after
factory.stop();
factory = new KafkaLookupExtractorFactory(bootstrapServers, topic, kafkaProperties);
boolean ok = factory.start();

Defensive patterns

Strategy: validation

Validate before calling

if (factory.isStarted()) return; // or rebuild factory before calling start()
boolean ok = factory.start();
if (!ok) { factory = new KafkaLookupExtractorFactory(bootstrap, topic, props); ok = factory.start(); }

Prevention

Treat factory instances as single-use: new instance per lifecycle
Check the boolean return of start() rather than assuming success
Never call stop() for transient suspensions

When it happens

Trigger: Calling start() after stop() was called on the same KafkaLookupExtractorFactory instance; the executorService is in shutdown state, so start() returns false.

Common situations: Lookup manager re-initializing a cached lookup instance that was previously torn down; restarting a suspended task after the lookup factory was closed; mismanaged lifecycle where the same factory object is reused across start/stop cycles.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/ba5363202a06335d. Report an issue: GitHub.