apache/kafka · error · KafkaException

The consumer background thread is not running and cannot pro

Error message

The consumer background thread is not running and cannot process requests.

What it means

Thrown by ApplicationEventHandler.ensureNetworkThreadAlive() when the async consumer's background network thread is null or no longer alive. The new (threaded) consumer delegates all work to that thread via an event queue; if the thread has terminated it will never drain the queue, so any further consumer call would either block indefinitely or surface a misleading timeout. This check fails fast with a clear message instead. Note the code comment explicitly calls the check racy-by-design — the thread could die right after the check, but the subsequent queue add() would then detect it.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/ApplicationEventHandler.java:176

                () -> log.warn("The application event handler was already closed")
        );
    }

    /**
     * Best-effort check that the consumer network thread is still alive. If the thread has
     * already terminated (due to a failure or shutdown), it will never process any events from
     * the queue. Rather than blocking indefinitely or timing out with a misleading error, this
     * fails fast with a clear error message.
     *
     * <p>Note: this is inherently racy — the thread could die between this check and the
     * subsequent {@code applicationEventQueue.add()}. That narrow window is acceptable because
     * any subsequent call to {@code add()} will detect the dead thread immediately.
     *
     * @throws KafkaException if the background thread is not alive
     */
    private void ensureNetworkThreadAlive() {
        if (networkThread == null || !networkThread.isAlive()) {
            throw new KafkaException(
                "The consumer background thread is not running and cannot process requests.");
        }
    }
}

View on GitHub (pinned to c31c9215e1)

Solutions

1. Do not reuse the KafkaConsumer after calling close() — create a fresh instance.
2. Inspect logs immediately preceding this error for the original Throwable that killed the network thread (it is logged separately, not nested here) and address that root cause.
3. Ensure consumer operations are only invoked from a single, owned thread and that no external code calls Thread.interrupt() on the network thread.
4. If running under a container/executor with tight thread limits, raise the thread ceiling so the network thread can actually start.
5. Wrap consumer usage in try-with-resources and on this exception discard the instance and rebuild it with backoff.

Example fix

// before
KafkaConsumer<String,String> c = new KafkaConsumer<>(props);
c.subscribe(singleton("t"));
c.close();
c.poll(Duration.ZERO); // -> KafkaException: background thread not running

// after
try (KafkaConsumer<String,String> c = new KafkaConsumer<>(props)) {
    c.subscribe(singleton("t"));
    c.poll(Duration.ofMillis(500));
}
// if a poll ever throws this, build a NEW consumer instance rather than retrying on the dead one.

// No reliable public API exposes background-thread liveness (the check is
// inherently racy by design, see ApplicationEventHandler.ensureNetworkThreadAlive).
// Defensive option: track the consumer lifecycle in your own code so you never
// call poll/commit after close().
private volatile boolean consumerClosed = false;

public void safePoll(KafkaConsumer<String,String> c, Duration timeout) {
    if (consumerClosed || c == null) return;
    c.poll(timeout);
}

@Override public void close() { consumerClosed = true; /* close consumer */ }

// The background (network) thread died or the consumer was closed. Treat as
// fatal for THIS consumer instance: catch, close, and rebuild a new consumer.
try {
    consumer.poll(Duration.ofMillis(500));
} catch (org.apache.kafka.common.KafkaException e) {
    if (e.getMessage() != null && e.getMessage().startsWith(
            "The consumer background thread is not running")) {
        log.error("Consumer network thread died, recreating consumer", e);
        closeQuietly(consumer);
        consumer = buildConsumer();   // fresh KafkaConsumer with same props
    } else {
        throw e;
    }
}

When it happens

Trigger: Any consumer operation that goes through ApplicationEventHandler.add() (poll, commitSync/Async, subscribe, unsubscribe, seek, position, offsetsForTimes, listOffsets, etc.) after the network thread has died or before/after close(). Triggered specifically when networkThread == null || !networkThread.isAlive().

Common situations: Reusing a KafkaConsumer instance after close() has been called; an uncaught exception or OutOfMemoryError killing the network thread silently; JVM thread limits preventing the thread from starting; calling consumer methods from a different thread after the consumer's owning thread was interrupted; framework code (Spring @KafkaListener, Vert.x) that pools/rewires consumer instances.

Related errors

• The timeout cannot be negative.
• Failed to close kafka consumer
• This consumer has already been closed.
• Topic collection to subscribe to cannot be null
• Topic collection to subscribe to cannot contain null or empt

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/8a594db309a7a005.json. Report an issue: GitHub.