apache/pulsar · error · RestException
Time-out while checking authorization
Error message
Time-out while checking authorization
What it means
During validateProducePermission the broker waits (bounded by metadataStoreOperationTimeoutSeconds) for the asynchronous authorization check against the metadata store; on TimeoutException it returns 500 with this message. It means the authorization decision could not be made in time.
Source
Thrown at pulsar-broker/src/main/java/org/apache/pulsar/broker/rest/TopicsBase.java:710
public void validateProducePermission() throws Exception {
if (pulsar().getConfiguration().isAuthenticationEnabled()
&& pulsar().getBrokerService().isAuthorizationEnabled()) {
if (!isClientAuthenticated(clientAppId())) {
throw new RestException(Status.UNAUTHORIZED, "Need to authenticate to perform the request");
}
AuthenticationParameters authParams = authParams();
boolean isAuthorized;
try {
isAuthorized = pulsar().getBrokerService().getAuthorizationService()
.allowTopicOperationAsync(topicName, TopicOperation.PRODUCE, authParams)
.get(config().getMetadataStoreOperationTimeoutSeconds(), SECONDS);
} catch (TimeoutException e) {
log.warn()
.attr("timeoutSec", config().getMetadataStoreOperationTimeoutSeconds())
.attr("topic", topicName)
.log("Timeout while checking authorization");
throw new RestException(Status.INTERNAL_SERVER_ERROR, "Time-out while checking authorization");
} catch (Exception e) {
log.warn()
.attr("role", authParams.getClientRole())
.attr("originalPrincipal", authParams.getOriginalPrincipal())
.attr("topic", topicName)
.exceptionMessage(e)
.log("Producer-client with Role - failed to get permissions for topic - .");
throw new RestException(Status.INTERNAL_SERVER_ERROR, "Failed to get permissions");
}
if (!isAuthorized) {
throw new RestException(Status.UNAUTHORIZED, "Unauthorized to produce to topic " + topicName);
}
}
}
}
View on GitHub (pinned to 820761864e)
Solutions
- Check metadata store health and latency (ZooKeeper/etcd metrics, network between broker and store)
- Increase metadataStoreOperationTimeoutSeconds in broker.conf if the store is legitimately slow
- Retry the produce request once the store recovers
- Scale/repair the metadata store ensemble if timeouts are recurrent
Example fix
// before (broker.conf) metadataStoreOperationTimeoutSeconds=30 # too tight for loaded ZK // after metadataStoreOperationTimeoutSeconds=120
Defensive patterns
Strategy: retry
Validate before calling
// no client pre-check; verify metadata store reachability // e.g. ping the ZooKeeper/etcd endpoint before heavy produce bursts
Try / catch
try {
produceViaRest(topic, payload);
} catch (RestException e) {
if (e.getResponse().getStatus() == 500 && e.getMessage().contains("Time-out while checking authorization")) {
backoffAndRetry(topic, payload, 3);
} else throw e;
} Prevention
- Monitor metadata store latency and set metadataStoreOperationTimeoutSeconds accordingly
- Keep authorization policy data lean to speed checks
- Alert on ZK/etcd health before broker-side timeouts occur
When it happens
Trigger: AuthorizationService.allowTopicOperationAsync(PRODUCE) doesn't complete within config().getMetadataStoreOperationTimeoutSeconds() — slow/overloaded metadata store (ZooKeeper/etcd), network partitions, or large permission sets.
Common situations: ZooKeeper latency/GC pauses under load; metadata store network issues; aggressive timeout configuration in busy clusters.
Related errors
- RestException(e)
- Failed to get permissions
- Invalid combination of Original principal cannot be empty if
- Proxy not authorized for super-user operation (proxy:%s)
- Original principal not authorized for super-user operation (
AI-assisted analysis of apache/pulsar@820761864e (2026-09-06).
Data as JSON: /api/errors/49e9aebc4c9c6f0a.
Report an issue: GitHub.