{"record":{"id":"0aeed64e533c51c1","repo":"apache/pulsar","slug":"rebalance-already-in-progress-0aeed6","errorCode":null,"errorMessage":"Rebalance already in progress","messagePattern":"Rebalance already in progress","errorType":"http","errorClass":"RestException","httpStatus":400,"severity":"warning","filePath":"pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/rest/api/WorkerImpl.java","lineNumber":230,"sourceCode":"        if (!isWorkerServiceAvailable()) {\n            throwUnavailableException();\n        }\n        throwIfNotSuperUser(authParams, \"get list of connectors\");\n        return this.worker().getConnectorsManager().getConnectorDefinitions();\n    }\n\n    @Override\n    public void rebalance(final URI uri, final AuthenticationParameters authParams) {\n        if (!isWorkerServiceAvailable()) {\n            throwUnavailableException();\n        }\n        throwIfNotSuperUser(authParams, \"rebalance cluster\");\n\n        if (worker().getLeaderService().isLeader()) {\n            try {\n                worker().getSchedulerManager().rebalanceIfNotInprogress();\n            } catch (SchedulerManager.RebalanceInProgressException e) {\n                throw new RestException(Status.BAD_REQUEST, \"Rebalance already in progress\");\n            } catch (SchedulerManager.TooFewWorkersException e) {\n                throw new RestException(Status.BAD_REQUEST, \"Too few workers (need at least 2)\");\n            }\n        } else {\n            WorkerInfo workerInfo = worker().getMembershipManager().getLeader();\n            if (workerInfo == null) {\n                throw new RestException(Status.INTERNAL_SERVER_ERROR, \"Leader cannot be determined\");\n            }\n            URI redirect =\n                    UriBuilder.fromUri(uri).host(workerInfo.getWorkerHostname()).port(workerInfo.getPort()).build();\n            throw new WebApplicationException(Response.temporaryRedirect(redirect).build());\n        }\n    }\n\n    @Override\n    public void drain(final URI uri, final String inWorkerId, final AuthenticationParameters authParams,\n                      boolean calledOnLeaderUri) {\n        if (!isWorkerServiceAvailable()) {","sourceCodeStart":212,"sourceCodeEnd":248,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/rest/api/WorkerImpl.java#L212-L248","documentation":"This HTTP 400 error is thrown by the Pulsar Functions worker's rebalance REST endpoint when another cluster-wide rebalance is already running. SchedulerManager.rebalanceIfNotInprogress() rejects concurrent rebalances to avoid overlapping rescheduling of function instances across workers, and WorkerImpl translates the RebalanceInProgressException into a RestException.","triggerScenarios":"Calling POST /admin/v2/worker/rebalance (via the REST API or pulsar-admin functions worker rebalance) while a prior rebalance triggered on the leader worker has not yet completed.","commonSituations":"Operators or automation scripts issuing rebalance requests concurrently (e.g. a scheduled job plus a manual run, or double-clicking a dashboard button); slow rebalances on large clusters make overlapping calls likely.","solutions":["Wait for the in-progress rebalance to finish, then re-issue the request","Check worker logs (SchedulerManager) for rebalance start/completion to confirm it finished before retrying","Remove duplicate/scheduled rebalance triggers so only one caller initiates it"],"exampleFix":"// before: blind retry\nadmin.functions().rebalance();\n// after: catch and retry later\ntry {\n    admin.functions().rebalance();\n} catch (PulsarAdminException e) {\n    if (e.getStatusCode() == 400 && e.getMessage().contains(\"Rebalance already in progress\")) {\n        // wait for current rebalance to complete, then retry\n    }\n}","handlingStrategy":"try-catch","validationCode":"List<WorkerInfo> workers = admin.functions().getCluster(); // ensure cluster healthy before rebalancing","typeGuard":null,"tryCatchPattern":"try {\n    admin.functions().rebalance();\n} catch (PulsarAdminException e) {\n    if (e.getStatusCode() == 400 && e.getMessage().contains(\"Rebalance already in progress\")) {\n        // back off and retry after the current rebalance completes\n    }\n}","preventionTips":["Serialize rebalance calls in automation (avoid concurrent/scheduled triggers)","Check worker logs or status before issuing a new rebalance","Use exponential backoff on 400 rebalance-in-progress responses"],"tags":["rest-api","concurrency","functions-worker"],"backgroundTag":"rebalance-in-progress","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}