{"record":{"id":"d7c1c23ceb311466","repo":"apache/pulsar","slug":"leader-not-yet-ready-please-retry-again","errorCode":null,"errorMessage":"Leader not yet ready. Please retry again","messagePattern":"Leader not yet ready\\. Please retry again","errorType":"http","errorClass":"RestException","httpStatus":503,"severity":"warning","filePath":"pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/rest/api/FunctionsImpl.java","lineNumber":734,"sourceCode":"            throw new RestException(Response.Status.BAD_REQUEST, \"Namespace is not provided\");\n        }\n        if (functionName == null) {\n            throw new RestException(Response.Status.BAD_REQUEST, \"Function name is not provided\");\n        }\n        FunctionMetaData functionMetaData;\n        try {\n            byte[] data = uploadedInputStream.readAllBytes();\n            functionMetaData = new FunctionMetaData();\n            functionMetaData.parseFrom(data);\n        } catch (IOException e) {\n            throw new RestException(Response.Status.BAD_REQUEST, \"Corrupt Function MetaData\");\n        }\n\n        // Redirect if we are not the leader\n        if (!worker().getLeaderService().isLeader()) {\n            WorkerInfo workerInfo = worker().getMembershipManager().getLeader();\n            if (workerInfo == null || workerInfo.getWorkerId().equals(worker().getWorkerConfig().getWorkerId())) {\n                throw new RestException(Response.Status.SERVICE_UNAVAILABLE,\n                        \"Leader not yet ready. Please retry again\");\n            }\n            URI redirect = UriBuilder.fromUri(uri).host(workerInfo.getWorkerHostname())\n                    .port(workerInfo.getPort()).build();\n            throw new WebApplicationException(Response.temporaryRedirect(redirect).build());\n        }\n\n        // Its possible that we are not the leader anymore. That will be taken care of by FunctionMetaDataManager\n        FunctionMetaDataManager functionMetaDataManager = worker().getFunctionMetaDataManager();\n        try {\n            functionMetaDataManager.updateFunctionOnLeader(functionMetaData, delete);\n        } catch (IllegalStateException e) {\n            throw new RestException(Response.Status.INTERNAL_SERVER_ERROR, e.getMessage());\n        } catch (IllegalArgumentException e) {\n            throw new RestException(Response.Status.BAD_REQUEST, e.getMessage());\n        }\n    }\n","sourceCodeStart":716,"sourceCodeEnd":752,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/rest/api/FunctionsImpl.java#L716-L752","documentation":"HTTP 503 thrown by updateFunctionOnWorkerLeader when this worker is not the leader and either no leader is currently known (workerInfo == null) or the 'leader' recorded is this very worker (membership state briefly inconsistent). The worker cannot serve the update and cannot redirect, so it asks the client to retry.","triggerScenarios":"POSTing an updateOnLeader request to a follower worker during leader election, right after the previous leader crashed, or during a cluster split where the membership manager has no fresh leader entry.","commonSituations":"Hitting a non-leader worker directly because a load balancer routed the request there; submitting function updates immediately after a worker restart or rolling upgrade; ZooKeeper/metadata-store hiccup delaying leader publication; small clusters where leadership is still being established at startup.","solutions":["Retry the request with backoff until a leader is elected (the message explicitly says 'Please retry again').","Query the cluster's current leader via the /leaderBroker or worker membership endpoints and route the update there.","Check worker logs and metadata-store connectivity if the condition persists beyond a few seconds."],"exampleFix":"// before\nResponse resp = client.post(updateUrl, body); // fails with 503 during election\n// after\nResponse resp;\nint attempts = 0;\ndo {\n    resp = client.post(updateUrl, body);\n    if (resp.status() == 503) {\n        Thread.sleep(1000L * ++attempts);\n    }\n} while (resp.status() == 503 && attempts < 5);","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    client.post(updateUrl, body);\n} catch (ApiException e) {\n    if (e.code() == 503 && e.body().contains(\"Leader not yet ready\")) {\n        sleepWithBackoff();\n        retryUpTo(5);\n    }\n}","preventionTips":["Implement automatic retry with exponential backoff for 503 on worker APIs.","Route writes through a leader-aware proxy or resolve the leader first via the membership endpoint.","Monitor leader-election duration so clients don't hammer workers during elections."],"tags":["rest","service-unavailable","leader-election","pulsar-functions","retryable"],"backgroundTag":"leader-not-available","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}