apache/pulsar · critical · RestException

Failed to update clusters because failed to create admin cli

Error message

Failed to update clusters because failed to create admin client for cluster ${remoteCluster}: ${e.getMessage()}

What it means

This 500 INTERNAL_SERVER_ERROR is thrown when validateClusterPairCompatibility successfully finds the remote cluster's ClusterData but pulsar().getBrokerService().getClusterPulsarAdmin(remoteCluster, ...) throws while constructing the PulsarAdmin client used to query the remote broker. The replication validation is aborted because the broker cannot open an admin connection to the remote cluster.

Source

Thrown at pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java:966

        return FutureUtil.waitForAll(validationFutures);
    }

    /**
     * Validates compatibility between the local cluster and a remote cluster.
     */
    private CompletableFuture<Void> validateClusterPairCompatibility(String localCluster, String remoteCluster) {
        return clusterResources().getClusterAsync(remoteCluster)
                .thenCompose(clusterDataOpt -> {
                    if (clusterDataOpt.isEmpty()) {
                        throw new RestException(Status.NOT_FOUND, "Cluster " + remoteCluster + " does not exist");
                    }
                    ClusterData clusterData = clusterDataOpt.get();
                    PulsarAdmin remoteAdmin;
                    try {
                        remoteAdmin = pulsar().getBrokerService()
                                .getClusterPulsarAdmin(remoteCluster, Optional.of(clusterData));
                    } catch (Exception e) {
                        throw new RestException(Status.INTERNAL_SERVER_ERROR,
                            "Failed to update clusters because failed to create admin client for cluster "
                            + remoteCluster + ": " + e.getMessage());
                    }

                    // If the local cluster and the target cluster are sharing ZK, the target cluster cannot create any
                    // topic before enabling replication, so verification can be skipped.
                    CompletableFuture<Policies> remoteNsPoliciesFuture = new CompletableFuture<>();
                    remoteAdmin.namespaces().getPoliciesAsync(namespaceName.toString())
                        .handle((v, ex) -> {
                            if (ex == null) {
                                remoteNsPoliciesFuture.complete(v);
                                return null;
                            }
                            // If namespace doesn't have override, return null.
                            Throwable actEx = FutureUtil.unwrapCompletionException(ex);
                            if (actEx instanceof PulsarAdminException.NotFoundException) {
                                remoteNsPoliciesFuture.completeExceptionally(new RestException(Status.CONFLICT,
                                    String.format("Failed to check auto-topic creation policy for"

View on GitHub (pinned to 820761864e)

Solutions

Verify the remote cluster's serviceUrl with GET /admin/v2/clusters/{cluster} and fix it via PUT /admin/v2/clusters/{cluster} (correct host, port, http/https scheme)
Test connectivity from the local broker host: curl the remote cluster's serviceUrl/admin/v2/clusters to confirm reachability and auth
Check broker logs for the underlying exception message appended to this error to identify whether it's DNS, TLS, or auth
Align TLS and auth configuration (tlsEnabled, brokerClientTlsEnabled, authPlugin/authParams) in ClusterData with the remote cluster's actual setup

Example fix

// before: cluster registered with wrong URL
ClusterData data = ClusterData.builder().serviceUrl("http://wrong-host:8080").build();
// after
ClusterData data = ClusterData.builder().serviceUrl("http://pulsar-us-west:8080").build();
admin.clusters().updateCluster("us-west", data);

Defensive patterns

Strategy: validation

Validate before calling

ClusterData cd = admin.clusters().getCluster(remoteCluster);
HttpURLConnection c = (HttpURLConnection) new URL(cd.getServiceUrl() + "/admin/v2/clusters").openConnection();
c.setConnectTimeout(3000);
if (c.getResponseCode() != 200) throw new IllegalStateException("Remote admin unreachable: " + cd.getServiceUrl());

Try / catch

try {
    admin.namespaces().setNamespaceReplicationClusters(ns, clusters);
} catch (PulsarAdminException.ServerSideErrorException e) {
    log.error("Admin client creation for remote cluster failed", e);
}

Prevention

Keep ClusterData serviceUrl and TLS/auth settings accurate and updated via config management
Test inter-cluster admin connectivity after every cluster URL or certificate change
Include replication health checks in monitoring (periodic getClusterData round-trips)

When it happens

Trigger: Calling namespace replication validation/update APIs where the remote cluster's serviceUrl is wrong, unreachable, malformed, or TLS configuration mismatches, causing the PulsarAdmin client construction (or its provider lookup) to fail.

Common situations: Cluster registered with a stale or wrong serviceUrl (hostname not resolvable, wrong port); TLS enabled on one side but not the other; auth plugin/parameter misconfiguration in ClusterData; DNS or firewall blocking the remote admin port.

Related errors

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/863de31d2182837c. Report an issue: GitHub.