{"record":{"id":"49e9aebc4c9c6f0a","repo":"apache/pulsar","slug":"time-out-while-checking-authorization","errorCode":null,"errorMessage":"Time-out while checking authorization","messagePattern":"Time-out while checking authorization","errorType":"http","errorClass":"RestException","httpStatus":500,"severity":"error","filePath":"pulsar-broker/src/main/java/org/apache/pulsar/broker/rest/TopicsBase.java","lineNumber":710,"sourceCode":"\n    public void validateProducePermission() throws Exception {\n        if (pulsar().getConfiguration().isAuthenticationEnabled()\n                && pulsar().getBrokerService().isAuthorizationEnabled()) {\n            if (!isClientAuthenticated(clientAppId())) {\n                throw new RestException(Status.UNAUTHORIZED, \"Need to authenticate to perform the request\");\n            }\n            AuthenticationParameters authParams = authParams();\n            boolean isAuthorized;\n            try {\n                isAuthorized = pulsar().getBrokerService().getAuthorizationService()\n                        .allowTopicOperationAsync(topicName, TopicOperation.PRODUCE, authParams)\n                        .get(config().getMetadataStoreOperationTimeoutSeconds(), SECONDS);\n            } catch (TimeoutException e) {\n                log.warn()\n                        .attr(\"timeoutSec\", config().getMetadataStoreOperationTimeoutSeconds())\n                        .attr(\"topic\", topicName)\n                        .log(\"Timeout while checking authorization\");\n                throw new RestException(Status.INTERNAL_SERVER_ERROR, \"Time-out while checking authorization\");\n            } catch (Exception e) {\n                log.warn()\n                        .attr(\"role\", authParams.getClientRole())\n                        .attr(\"originalPrincipal\", authParams.getOriginalPrincipal())\n                        .attr(\"topic\", topicName)\n                        .exceptionMessage(e)\n                        .log(\"Producer-client with Role - failed to get permissions for topic - .\");\n                throw new RestException(Status.INTERNAL_SERVER_ERROR, \"Failed to get permissions\");\n            }\n\n            if (!isAuthorized) {\n                throw new RestException(Status.UNAUTHORIZED, \"Unauthorized to produce to topic \" + topicName);\n            }\n        }\n    }\n\n}\n","sourceCodeStart":692,"sourceCodeEnd":728,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/pulsar-broker/src/main/java/org/apache/pulsar/broker/rest/TopicsBase.java#L692-L728","documentation":"During validateProducePermission the broker waits (bounded by metadataStoreOperationTimeoutSeconds) for the asynchronous authorization check against the metadata store; on TimeoutException it returns 500 with this message. It means the authorization decision could not be made in time.","triggerScenarios":"AuthorizationService.allowTopicOperationAsync(PRODUCE) doesn't complete within config().getMetadataStoreOperationTimeoutSeconds() — slow/overloaded metadata store (ZooKeeper/etcd), network partitions, or large permission sets.","commonSituations":"ZooKeeper latency/GC pauses under load; metadata store network issues; aggressive timeout configuration in busy clusters.","solutions":["Check metadata store health and latency (ZooKeeper/etcd metrics, network between broker and store)","Increase metadataStoreOperationTimeoutSeconds in broker.conf if the store is legitimately slow","Retry the produce request once the store recovers","Scale/repair the metadata store ensemble if timeouts are recurrent"],"exampleFix":"// before (broker.conf)\nmetadataStoreOperationTimeoutSeconds=30  # too tight for loaded ZK\n// after\nmetadataStoreOperationTimeoutSeconds=120","handlingStrategy":"retry","validationCode":"// no client pre-check; verify metadata store reachability\n// e.g. ping the ZooKeeper/etcd endpoint before heavy produce bursts","typeGuard":null,"tryCatchPattern":"try {\n    produceViaRest(topic, payload);\n} catch (RestException e) {\n    if (e.getResponse().getStatus() == 500 && e.getMessage().contains(\"Time-out while checking authorization\")) {\n        backoffAndRetry(topic, payload, 3);\n    } else throw e;\n}","preventionTips":["Monitor metadata store latency and set metadataStoreOperationTimeoutSeconds accordingly","Keep authorization policy data lean to speed checks","Alert on ZK/etcd health before broker-side timeouts occur"],"tags":["pulsar","rest-api","authorization","timeout","metadata-store"],"backgroundTag":"metadata-store-timeout","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}