{"record":{"id":"ed4f479ba6faaf1e","repo":"apache/pulsar","slug":"failed-to-create-statefulset-for-function-s","errorCode":null,"errorMessage":"Failed to create statefulset for function %s","messagePattern":"Failed to create statefulset for function (.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"critical","filePath":"pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/kubernetes/KubernetesRuntime.java","lineNumber":595,"sourceCode":"                                .success(false)\n                                .errorMsg(errorMsg)\n                                .build();\n                    }\n\n                    return Actions.ActionResult.builder().success(true).build();\n                })\n                .build();\n\n\n        AtomicBoolean success = new AtomicBoolean(false);\n        Actions.newBuilder()\n                .addAction(createStatefulSet.toBuilder()\n                        .onSuccess(ignored -> success.set(true))\n                        .build())\n                .run();\n\n        if (!success.get()) {\n            throw new RuntimeException(String.format(\"Failed to create statefulset for function %s\", fqfn));\n        }\n    }\n\n\n    public void deleteStatefulSet() throws InterruptedException {\n        String statefulSetName = createJobName(instanceConfig.getFunctionDetails(), this.jobName);\n\n        String fqfn = FunctionCommon.getFullyQualifiedName(instanceConfig.getFunctionDetails());\n        Actions.Action deleteStatefulSet = Actions.Action.builder()\n                .actionName(String.format(\"Deleting statefulset for function %s\", fqfn))\n                .numRetries(KubernetesRuntimeFactory.numRetries)\n                .sleepBetweenInvocationsMs(KubernetesRuntimeFactory.sleepBetweenRetriesMs)\n                .supplier(() -> {\n                    try {\n                        appsClient.deleteNamespacedStatefulSet(\n                                statefulSetName,\n                                jobNamespace)\n                                .gracePeriodSeconds(gracePeriodSeconds)","sourceCodeStart":577,"sourceCodeEnd":613,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/kubernetes/KubernetesRuntime.java#L577-L613","documentation":"KubernetesRuntime.submitStatefulSet() runs a retry Action to create the function's StatefulSet; if it does not succeed after retries, it throws RuntimeException naming the function. The function's pods cannot be scheduled without the StatefulSet.","triggerScenarios":"submitStatefulSet() called from start() when creating the StatefulSet via the k8s client fails on every attempt — API errors, invalid spec, quota exceeded, or RBAC denial.","commonSituations":"Resource quota exceeded in the namespace; invalid image name or pull secret; worker ServiceAccount lacking StatefulSet create permissions; API server connectivity problems.","solutions":["Inspect the logged cause from the k8s client (connection, 403, 422 validation, quota).","Verify RBAC allows creating statefulsets in the target namespace.","Check namespace ResourceQuota and pod limits.","Validate the function's image name, pull policy and secrets referenced in the statefulset spec."],"exampleFix":"// before: quota exceeded\nkubectl describe resourcequota -n <ns>\n// after: raise quota or reduce requested cpu/memory in function config","handlingStrategy":"retry","validationCode":"// before starting: check quota and RBAC\nkubectl auth can-i create statefulsets.apps -n <ns> --as=system:serviceaccount:<ns>:<worker-sa>\nkubectl describe resourcequota -n <ns>","typeGuard":null,"tryCatchPattern":"try {\n    kubernetesRuntime.start();\n} catch (RuntimeException e) {\n    if (e.getMessage().startsWith(\"Failed to create statefulset for function\")) {\n        log.error(\"StatefulSet creation failed; check quota, image validity, and RBAC\", e);\n    }\n    throw e;\n}","preventionTips":["Validate container images and pull secrets referenced by functions exist in the cluster","Set realistic CPU/memory requests within namespace quotas","Grant the worker ServiceAccount RBAC for statefulsets","Ensure the k8s client config/credentials on the worker are valid"],"tags":["kubernetes","functions-runtime","statefulset"],"backgroundTag":"kubernetes-resource-creation-failed","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}