apache/pulsar · warning · RestException

Rebalance already in progress

Error message

Rebalance already in progress

What it means

This HTTP 400 error is thrown by the Pulsar Functions worker's rebalance REST endpoint when another cluster-wide rebalance is already running. SchedulerManager.rebalanceIfNotInprogress() rejects concurrent rebalances to avoid overlapping rescheduling of function instances across workers, and WorkerImpl translates the RebalanceInProgressException into a RestException.

Source

Thrown at pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/rest/api/WorkerImpl.java:230

        if (!isWorkerServiceAvailable()) {
            throwUnavailableException();
        }
        throwIfNotSuperUser(authParams, "get list of connectors");
        return this.worker().getConnectorsManager().getConnectorDefinitions();
    }

    @Override
    public void rebalance(final URI uri, final AuthenticationParameters authParams) {
        if (!isWorkerServiceAvailable()) {
            throwUnavailableException();
        }
        throwIfNotSuperUser(authParams, "rebalance cluster");

        if (worker().getLeaderService().isLeader()) {
            try {
                worker().getSchedulerManager().rebalanceIfNotInprogress();
            } catch (SchedulerManager.RebalanceInProgressException e) {
                throw new RestException(Status.BAD_REQUEST, "Rebalance already in progress");
            } catch (SchedulerManager.TooFewWorkersException e) {
                throw new RestException(Status.BAD_REQUEST, "Too few workers (need at least 2)");
            }
        } else {
            WorkerInfo workerInfo = worker().getMembershipManager().getLeader();
            if (workerInfo == null) {
                throw new RestException(Status.INTERNAL_SERVER_ERROR, "Leader cannot be determined");
            }
            URI redirect =
                    UriBuilder.fromUri(uri).host(workerInfo.getWorkerHostname()).port(workerInfo.getPort()).build();
            throw new WebApplicationException(Response.temporaryRedirect(redirect).build());
        }
    }

    @Override
    public void drain(final URI uri, final String inWorkerId, final AuthenticationParameters authParams,
                      boolean calledOnLeaderUri) {
        if (!isWorkerServiceAvailable()) {

View on GitHub (pinned to 820761864e)

Solutions

1. Wait for the in-progress rebalance to finish, then re-issue the request
2. Check worker logs (SchedulerManager) for rebalance start/completion to confirm it finished before retrying
3. Remove duplicate/scheduled rebalance triggers so only one caller initiates it

Example fix

// before: blind retry
admin.functions().rebalance();
// after: catch and retry later
try {
    admin.functions().rebalance();
} catch (PulsarAdminException e) {
    if (e.getStatusCode() == 400 && e.getMessage().contains("Rebalance already in progress")) {
        // wait for current rebalance to complete, then retry
    }
}

List<WorkerInfo> workers = admin.functions().getCluster(); // ensure cluster healthy before rebalancing

try {
    admin.functions().rebalance();
} catch (PulsarAdminException e) {
    if (e.getStatusCode() == 400 && e.getMessage().contains("Rebalance already in progress")) {
        // back off and retry after the current rebalance completes
    }
}

When it happens

Trigger: Calling POST /admin/v2/worker/rebalance (via the REST API or pulsar-admin functions worker rebalance) while a prior rebalance triggered on the leader worker has not yet completed.

Common situations: Operators or automation scripts issuing rebalance requests concurrently (e.g. a scheduled job plus a manual run, or double-clicking a dashboard button); slow rebalances on large clusters make overlapping calls likely.

Related errors

• Concurrent modification
• Too few workers (need at least 2)
• Another drain is in progress
• cluster data is required
• Cluster already exists

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/0aeed64e533c51c1. Report an issue: GitHub.