{"record":{"id":"863de31d2182837c","repo":"apache/pulsar","slug":"failed-to-update-clusters-because-failed-to-create","errorCode":null,"errorMessage":"Failed to update clusters because failed to create admin client for cluster ${remoteCluster}: ${e.getMessage()}","messagePattern":"Failed to update clusters because failed to create admin client for cluster (.+?): (.+?)","errorType":"http","errorClass":"RestException","httpStatus":500,"severity":"critical","filePath":"pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java","lineNumber":966,"sourceCode":"        return FutureUtil.waitForAll(validationFutures);\n    }\n\n    /**\n     * Validates compatibility between the local cluster and a remote cluster.\n     */\n    private CompletableFuture<Void> validateClusterPairCompatibility(String localCluster, String remoteCluster) {\n        return clusterResources().getClusterAsync(remoteCluster)\n                .thenCompose(clusterDataOpt -> {\n                    if (clusterDataOpt.isEmpty()) {\n                        throw new RestException(Status.NOT_FOUND, \"Cluster \" + remoteCluster + \" does not exist\");\n                    }\n                    ClusterData clusterData = clusterDataOpt.get();\n                    PulsarAdmin remoteAdmin;\n                    try {\n                        remoteAdmin = pulsar().getBrokerService()\n                                .getClusterPulsarAdmin(remoteCluster, Optional.of(clusterData));\n                    } catch (Exception e) {\n                        throw new RestException(Status.INTERNAL_SERVER_ERROR,\n                            \"Failed to update clusters because failed to create admin client for cluster \"\n                            + remoteCluster + \": \" + e.getMessage());\n                    }\n\n                    // If the local cluster and the target cluster are sharing ZK, the target cluster cannot create any\n                    // topic before enabling replication, so verification can be skipped.\n                    CompletableFuture<Policies> remoteNsPoliciesFuture = new CompletableFuture<>();\n                    remoteAdmin.namespaces().getPoliciesAsync(namespaceName.toString())\n                        .handle((v, ex) -> {\n                            if (ex == null) {\n                                remoteNsPoliciesFuture.complete(v);\n                                return null;\n                            }\n                            // If namespace doesn't have override, return null.\n                            Throwable actEx = FutureUtil.unwrapCompletionException(ex);\n                            if (actEx instanceof PulsarAdminException.NotFoundException) {\n                                remoteNsPoliciesFuture.completeExceptionally(new RestException(Status.CONFLICT,\n                                    String.format(\"Failed to check auto-topic creation policy for\"","sourceCodeStart":948,"sourceCodeEnd":984,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java#L948-L984","documentation":"This 500 INTERNAL_SERVER_ERROR is thrown when validateClusterPairCompatibility successfully finds the remote cluster's ClusterData but pulsar().getBrokerService().getClusterPulsarAdmin(remoteCluster, ...) throws while constructing the PulsarAdmin client used to query the remote broker. The replication validation is aborted because the broker cannot open an admin connection to the remote cluster.","triggerScenarios":"Calling namespace replication validation/update APIs where the remote cluster's serviceUrl is wrong, unreachable, malformed, or TLS configuration mismatches, causing the PulsarAdmin client construction (or its provider lookup) to fail.","commonSituations":"Cluster registered with a stale or wrong serviceUrl (hostname not resolvable, wrong port); TLS enabled on one side but not the other; auth plugin/parameter misconfiguration in ClusterData; DNS or firewall blocking the remote admin port.","solutions":["Verify the remote cluster's serviceUrl with GET /admin/v2/clusters/{cluster} and fix it via PUT /admin/v2/clusters/{cluster} (correct host, port, http/https scheme)","Test connectivity from the local broker host: curl the remote cluster's serviceUrl/admin/v2/clusters to confirm reachability and auth","Check broker logs for the underlying exception message appended to this error to identify whether it's DNS, TLS, or auth","Align TLS and auth configuration (tlsEnabled, brokerClientTlsEnabled, authPlugin/authParams) in ClusterData with the remote cluster's actual setup"],"exampleFix":"// before: cluster registered with wrong URL\nClusterData data = ClusterData.builder().serviceUrl(\"http://wrong-host:8080\").build();\n// after\nClusterData data = ClusterData.builder().serviceUrl(\"http://pulsar-us-west:8080\").build();\nadmin.clusters().updateCluster(\"us-west\", data);","handlingStrategy":"validation","validationCode":"ClusterData cd = admin.clusters().getCluster(remoteCluster);\nHttpURLConnection c = (HttpURLConnection) new URL(cd.getServiceUrl() + \"/admin/v2/clusters\").openConnection();\nc.setConnectTimeout(3000);\nif (c.getResponseCode() != 200) throw new IllegalStateException(\"Remote admin unreachable: \" + cd.getServiceUrl());","typeGuard":null,"tryCatchPattern":"try {\n    admin.namespaces().setNamespaceReplicationClusters(ns, clusters);\n} catch (PulsarAdminException.ServerSideErrorException e) {\n    log.error(\"Admin client creation for remote cluster failed\", e);\n}","preventionTips":["Keep ClusterData serviceUrl and TLS/auth settings accurate and updated via config management","Test inter-cluster admin connectivity after every cluster URL or certificate change","Include replication health checks in monitoring (periodic getClusterData round-trips)"],"tags":["network","rest-api","admin-client","connectivity"],"backgroundTag":"admin-client-creation-failed","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}