{"id":"72eb846138339b99","repo":"apache/kafka","slug":"not-authorized-to-access-topics-unauthorizedtop","errorCode":null,"errorMessage":"Not authorized to access topics: ${unauthorizedTopics}","messagePattern":"Not authorized to access topics: (.+?)","errorType":"exception","errorClass":"TopicAuthorizationException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/clients/consumer/internals/TopicMetadataFetcher.java","lineNumber":113,"sourceCode":" // Save the round trip if no topics are requested.\n if (!request.isAllTopics() && request.emptyTopicList())\n return Collections.emptyMap();\n\n long attempts = 0L;\n do {\n RequestFuture future = sendMetadataRequest(request);\n client.poll(future, timer);\n\n if (future.failed() && !future.isRetriable())\n throw future.exception();\n\n if (future.succeeded()) {\n MetadataResponse response = (MetadataResponse) future.value().responseBody();\n Cluster cluster = response.buildCluster();\n\n Set unauthorizedTopics = cluster.unauthorizedTopics();\n if (!unauthorizedTopics.isEmpty())\n throw new TopicAuthorizationException(unauthorizedTopics);\n\n boolean shouldRetry = false;\n Map errors = response.errors();\n if (!errors.isEmpty()) {\n // if there were errors, we need to check whether they were fatal or whether\n // we should just retry\n\n log.debug(\"Topic metadata fetch included errors: {}\", errors);\n\n for (Map.Entry errorEntry : errors.entrySet()) {\n String topic = errorEntry.getKey();\n Errors error = errorEntry.getValue();\n\n if (error == Errors.INVALID_TOPIC_EXCEPTION)\n throw new InvalidTopicException(\"Topic '\" + topic + \"' is invalid\");\n else if (error == Errors.UNKNOWN_TOPIC_OR_PARTITION)\n // if a requested topic is unknown, we just continue and let it be absent\n // in the returned map","sourceCodeStart":95,"sourceCodeEnd":131,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/clients/consumer/internals/TopicMetadataFetcher.java#L95-L131","documentation":"Thrown as TopicAuthorizationException by the legacy/sync TopicMetadataFetcher.getTopicMetadata when the broker's MetadataResponse lists one or more topics in cluster.unauthorizedTopics(). The client surfaced it instead of silently dropping the topic because the principal authenticated correctly but lacks the Describe ACL on those topics. Most commonly raised from Consumer.partitionsFor() / listTopics() paths.","triggerScenarios":"Calling consumer.partitionsFor(\"topic\"), consumer.listTopics(), or any path that triggers a MetadataRequest where the authenticated principal lacks the DESCRIBE operation ACL on at least one requested topic. Also raised when a topic name is correct but the ACL was scoped to a different principal or prefixed differently.","commonSituations":"Service principals mismatched between producer and consumer (e.g. producer runs as user=alice, consumer as user=bob without a matching ACL); authorizer configured (SimpleAclAuthorizer / StandardAuthorizer in KRaft) but ACLs never granted; SASL/JAAS config wrong so the client authenticates as ANONYMOUS; cross-environment promotion without copying ACLs.","solutions":["Grant the DESCRIBE (and READ for consume) ACL: kafka-acls --add --allow-principal User:bob --operation Describe --operation Read --topic orders.","Verify the principal the consumer actually presents: enable DEBUG org.apache.kafka.clients.NetworkClient / check broker server.log for the PrincipalLoader output.","Confirm sasl.mechanism, sasl.jaas.config, and security.protocol match what the broker expects (mismatch silently authenticates as ANONYMOUS).","If using prefix ACLs, ensure the topic literal falls under the granted prefix (kafka-acls --resource-pattern-type prefixed)."],"exampleFix":"// before: consumer runs as wrong principal, no ACL\nbin/kafka-acls.sh --bootstrap-server broker:9092 \\\n --list --principal User:bob # (empty)\n\n// after: grant Describe + Read on the topic\nbin/kafka-acls.sh --bootstrap-server broker:9092 --add \\\n --allow-principal User:bob --operation Describe --operation Read \\\n --topic orders","handlingStrategy":"try-catch","validationCode":"// Client-side ACL preflight (optional, heavier): verify READ/DESCRIBE before subscribing.\ntry (var admin = org.apache.kafka.clients.admin.AdminClient.create(commonProps)) {\n java.util.Collection have =\n admin.describeAcls(org.apache.kafka.common.acl.AccessControlEntryFilter.ANY).values().get();\n // intersect with required (topic, READ/DESCRIBE) before consumer.subscribe(...)\n}","typeGuard":"import org.apache.kafka.common.errors.TopicAuthorizationException;\n\n/** Narrows a thrown Throwable to a TopicAuthorizationException and extracts unauthorized topic names. */\nstatic java.util.Optional> unauthorizedTopics(Throwable t) {\n if (t instanceof TopicAuthorizationException) {\n return java.util.Optional.of(((TopicAuthorizationException) t).unauthorizedTopics());\n }\n return java.util.Optional.empty();\n}","tryCatchPattern":"try {\n consumer.subscribe(singleton(topic));\n consumer.poll(Duration.ofMillis(1000));\n} catch (org.apache.kafka.common.errors.TopicAuthorizationException e) {\n java.util.Set unauthorized = e.unauthorizedTopics();\n log.error(\"Principal lacks READ/DESCRIBE on {}; provision ACLs via kafka-acls\", unauthorized);\n // surface to ops; do not blindly retry — this will not self-heal without ACL changes\n throw e;\n}","preventionTips":["Provision READ (and DESCRIBE) ACLs for the consumer principal before deploy, via kafka-acls --add.","Confirm the principal/clientId in JAAS or OAuth config actually matches the ACL principal.","Fail-fast on startup with an ACL preflight check rather than discovering it mid-poll.","Include unauthorized topic names in the error surfaced to operations for fast remediation."],"tags":["kafka","consumer","acl","authorization","metadata","sasl"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}