apache/pulsar · error · RestException

Time-out while checking authorization

Error message

Time-out while checking authorization

What it means

During validateProducePermission the broker waits (bounded by metadataStoreOperationTimeoutSeconds) for the asynchronous authorization check against the metadata store; on TimeoutException it returns 500 with this message. It means the authorization decision could not be made in time.

Source

Thrown at pulsar-broker/src/main/java/org/apache/pulsar/broker/rest/TopicsBase.java:710

    public void validateProducePermission() throws Exception {
        if (pulsar().getConfiguration().isAuthenticationEnabled()
                && pulsar().getBrokerService().isAuthorizationEnabled()) {
            if (!isClientAuthenticated(clientAppId())) {
                throw new RestException(Status.UNAUTHORIZED, "Need to authenticate to perform the request");
            }
            AuthenticationParameters authParams = authParams();
            boolean isAuthorized;
            try {
                isAuthorized = pulsar().getBrokerService().getAuthorizationService()
                        .allowTopicOperationAsync(topicName, TopicOperation.PRODUCE, authParams)
                        .get(config().getMetadataStoreOperationTimeoutSeconds(), SECONDS);
            } catch (TimeoutException e) {
                log.warn()
                        .attr("timeoutSec", config().getMetadataStoreOperationTimeoutSeconds())
                        .attr("topic", topicName)
                        .log("Timeout while checking authorization");
                throw new RestException(Status.INTERNAL_SERVER_ERROR, "Time-out while checking authorization");
            } catch (Exception e) {
                log.warn()
                        .attr("role", authParams.getClientRole())
                        .attr("originalPrincipal", authParams.getOriginalPrincipal())
                        .attr("topic", topicName)
                        .exceptionMessage(e)
                        .log("Producer-client with Role - failed to get permissions for topic - .");
                throw new RestException(Status.INTERNAL_SERVER_ERROR, "Failed to get permissions");
            }

            if (!isAuthorized) {
                throw new RestException(Status.UNAUTHORIZED, "Unauthorized to produce to topic " + topicName);
            }
        }
    }

}

View on GitHub (pinned to 820761864e)

Solutions

  1. Check metadata store health and latency (ZooKeeper/etcd metrics, network between broker and store)
  2. Increase metadataStoreOperationTimeoutSeconds in broker.conf if the store is legitimately slow
  3. Retry the produce request once the store recovers
  4. Scale/repair the metadata store ensemble if timeouts are recurrent

Example fix

// before (broker.conf)
metadataStoreOperationTimeoutSeconds=30  # too tight for loaded ZK
// after
metadataStoreOperationTimeoutSeconds=120
Defensive patterns

Strategy: retry

Validate before calling

// no client pre-check; verify metadata store reachability
// e.g. ping the ZooKeeper/etcd endpoint before heavy produce bursts

Try / catch

try {
    produceViaRest(topic, payload);
} catch (RestException e) {
    if (e.getResponse().getStatus() == 500 && e.getMessage().contains("Time-out while checking authorization")) {
        backoffAndRetry(topic, payload, 3);
    } else throw e;
}

Prevention

When it happens

Trigger: AuthorizationService.allowTopicOperationAsync(PRODUCE) doesn't complete within config().getMetadataStoreOperationTimeoutSeconds() — slow/overloaded metadata store (ZooKeeper/etcd), network partitions, or large permission sets.

Common situations: ZooKeeper latency/GC pauses under load; metadata store network issues; aggressive timeout configuration in busy clusters.

Related errors


AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/49e9aebc4c9c6f0a. Report an issue: GitHub.