apache/pulsar · error · PulsarClientException

Interrupted while waiting for layout update

Error message

Interrupted while waiting for layout update

What it means

When the target segment is gone (sealed by split/merge or topic migration), sendInternal drops the stale per-segment producer and sleeps with linear backoff (capped at SEND_RETRY_MAX_BACKOFF_MS) waiting for the DAG watch to deliver a new layout. If the sleeping thread is interrupted, the interrupt flag is restored and this PulsarClientException is thrown.

Source

Thrown at pulsar-client-v5/src/main/java/org/apache/pulsar/client/impl/v5/ScalableTopicProducer.java:234

            } catch (org.apache.pulsar.client.api.PulsarClientException e) {
                // Thrown by the v4 producer's send().
                if (!isSegmentGoneError(e)) {
                    throw new PulsarClientException(e.getMessage(), e);
                }
                lastError = new PulsarClientException(e.getMessage(), e);
            }
            // The target segment is gone: sealed by a split/merge, or terminated by a
            // regular-to-scalable migration. Drop the stale per-segment producer and wait
            // for the DAG watch to deliver the new layout; routeMessage on the next attempt
            // lands on an active child.
            log.info().attr("segmentId", segmentId).attr("attempt", attempt + 1)
                    .log("Target segment gone, waiting for layout update");
            segmentProducers.remove(segmentId);
            try {
                Thread.sleep(Math.min(100L * (attempt + 1), SEND_RETRY_MAX_BACKOFF_MS));
            } catch (InterruptedException ie) {
                Thread.currentThread().interrupt();
                throw new PulsarClientException("Interrupted while waiting for layout update", ie);
            }
        }
        throw lastError != null ? lastError
                : new PulsarClientException("Failed to send after segment termination retries");
    }

    /**
     * True if {@code t} (or one of its causes) signals that the target segment is gone —
     * sealed by a split/merge or terminated by a regular-to-scalable migration — so the send
     * should be retried once the new layout arrives. Handles both the v4 exceptions thrown by
     * {@code send()} and the V5-wrapped exceptions thrown while (re)creating the per-segment
     * producer on a now-terminated topic.
     */
    private static boolean isSegmentGoneError(Throwable t) {
        for (Throwable cause = t; cause != null; cause = cause.getCause()) {
            if (cause instanceof org.apache.pulsar.client.api.PulsarClientException.TopicTerminatedException) {
                return true;
            }

View on GitHub (pinned to 820761864e)

Solutions

1. Do not interrupt producer send threads; rely on producer.closeAsync() for cancellation.
2. If cancellation is required, catch this exception and re-send after the new layout is observed.
3. Check DAG watch/layout delivery latency if interruption timeouts are chronically hit during migrations.
4. Restore handling: the interrupt flag is re-set, so the surrounding task should terminate promptly.

Example fix

// before
Thread cancel: task.interrupt() while producer.send(msg) is retrying
// after
producer.closeAsync().thenRun(task::cancel); // cancel via producer lifecycle, not interrupt

try {
    producer.send(msg);
} catch (PulsarClientException e) {
    if (e.getCause() instanceof InterruptedException) {
        Thread.currentThread().interrupt();
        return; // abandon send, layout will settle
    }
    throw e;
}

When it happens

Trigger: Sending to a scalable topic right after a segment split/merge or regular-to-scalable migration, while the sending thread is interrupted during the layout-wait sleep.

Common situations: Shutdown or task cancellation exactly during a layout transition; test harnesses interrupting producer threads; slow DAG watch delivery combined with aggressive cancellation.

Related errors

• Interrupted initializing OAuth2 IdP TLS factory
• Close interrupted
• (wraps v4 producer send failure message)
• Interrupted while creating segment producer
• No active segments

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/d4b32da1e7c05fb7. Report an issue: GitHub.