apache/pulsar · warning · RestException

Segment topic not loaded: ${segmentTopic}

Error message

Segment topic not loaded: ${segmentTopic}

What it means

HTTP 404 thrown by the segment subscription-backlog endpoint when the segment topic is not currently loaded on a broker. The endpoint deliberately refuses to return 0 backlog for an unloaded topic, because that could be misread as 'segment drained'; a 404 tells the caller to retry once the topic loads.

Source

Thrown at pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/Segments.java:321

            @Parameter(description = "Segment descriptor (e.g. 0000-7fff-1)", required = true)
            @PathParam("descriptor") String descriptor,
            @Parameter(description = "Subscription name", required = true)
            @PathParam("subscription") String subscription,
            @Parameter(description = "Whether leader broker redirected this call to this broker.")
            @QueryParam("authoritative") @DefaultValue("false") boolean authoritative) {
        validateNamespaceName(tenant, namespace);
        TopicName segmentTopic = segmentTopicName(tenant, namespace, encodedTopic, descriptor);

        validateSuperUserAccessAsync()
                .thenCompose(__ -> validateTopicOwnershipAsync(segmentTopic, authoritative))
                .thenCompose(__ -> pulsar().getBrokerService().getTopicIfExists(segmentTopic.toString()))
                .thenAccept(optTopic -> {
                    if (optTopic.isEmpty()) {
                        // No topic loaded → no subscription cursor → no backlog. Returning
                        // 0 here would be wrong (caller might mark the segment drained on
                        // a topic that simply hasn't loaded yet); a 404 forces the caller
                        // to retry, which matches our drain-poll contract.
                        throw new RestException(Response.Status.NOT_FOUND,
                                "Segment topic not loaded: " + segmentTopic);
                    }
                    var sub = optTopic.get().getSubscription(subscription);
                    if (sub == null) {
                        throw new RestException(Response.Status.NOT_FOUND,
                                "Subscription not found on segment: " + subscription);
                    }
                    asyncResponse.resume(sub.getNumberOfEntriesInBacklog(false));
                })
                .exceptionally(ex -> {
                    log.error().attr("clientAppId", clientAppId()).attr("segment", segmentTopic)
                            .exception(ex).log("Failed to get segment subscription backlog");
                    resumeAsyncResponseExceptionally(asyncResponse, ex);
                    return null;
                });
    }

    @POST

View on GitHub (pinned to 820761864e)

Solutions

1. Make the drain-poll loop treat 404 as 'retry later' and re-poll after a delay.
2. Force the segment topic to load by connecting a reader/producer or an admin lookup before polling backlog.
3. Verify the segment still exists in the scalable topic metadata; remove it from the drain list if deleted.

Example fix

// before
const backlog = await getBacklog(seg); // throws 404 when unloaded
// after
let backlog;
try { backlog = await getBacklog(seg); }
catch (e) { if (e.status === 404) { await sleep(retryMs); continue; } throw e; }

const md = await admin.scalableTopics().getScalableTopicMetadataAsync(tenant, ns, parentTopic);
if (!md.getSegments().containsKey(segmentId)) throw new Error('unknown segment');

try {
  const backlog = await admin.scalableTopics().getSubscriptionBacklog(segmentTopic, sub);
} catch (e) {
  if (e.status === 404 && /not loaded/.test(e.message)) { await sleep(pollMs); return pollBacklog(segmentTopic, sub); }
  throw e;
}

When it happens

Trigger: Polling GET backlog for a segment whose topic has no active broker ownership — right after broker restart, before any client touched the segment (segments are not auto-created/loaded), or after the segment was deleted.

Common situations: Drain-monitoring loops polling segments that have never received traffic; querying backlog immediately after a broker failover before bundles reassign; polling a segment of an idle scalable topic where the broker unloaded it.

Related errors

• Subscription not found on segment: ${subscription}
• Segment topic not found: ${segmentTopic}
• Topic does not exist: ${persistentBase}
• Scalable topic not found: ${tn}
• Cannot terminate non-persistent topic: ${segmentTopic}

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/af0fe4e168fcf911. Report an issue: GitHub.