{"record":{"id":"4ddfc4623f171588","repo":"apache/pulsar","slug":"failed-to-upsert-authentication-secret-for-functio","errorCode":null,"errorMessage":"Failed to upsert authentication secret for function %s/%s/%s","messagePattern":"Failed to upsert authentication secret for function (.+?)/(.+?)/(.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/auth/KubernetesSecretsTokenAuthProvider.java","lineNumber":348,"sourceCode":"                        return Actions.ActionResult.builder()\n                                .success(false)\n                                .errorMsg(errorMsg)\n                                .build();\n                    }\n\n                    return Actions.ActionResult.builder().success(true).build();\n                })\n                .build();\n\n        AtomicBoolean success = new AtomicBoolean(false);\n        Actions.newBuilder()\n                .addAction(createAuthSecret.toBuilder()\n                        .onSuccess(ignore -> success.set(true))\n                        .build())\n                .run();\n\n        if (!success.get()) {\n            throw new RuntimeException(\n                    String.format(\"Failed to upsert authentication secret for function %s/%s/%s\", tenant, namespace,\n                            name));\n        }\n    }\n\n    private String createSecret(String token, FunctionDetails funcDetails)\n            throws ApiException, InterruptedException {\n        String kubeNamespace = getKubeNamespace(funcDetails);\n        String tenant = funcDetails.getTenant();\n        String namespace = funcDetails.getNamespace();\n        String name = funcDetails.getName();\n\n        StringBuilder sb = new StringBuilder();\n        Actions.Action createAuthSecret = Actions.Action.builder()\n                .actionName(\n                        String.format(\"Creating authentication secret for function %s/%s/%s\", tenant, namespace, name))\n                .numRetries(NUM_RETRIES)\n                .sleepBetweenInvocationsMs(SLEEP_BETWEEN_RETRIES_MS)","sourceCodeStart":330,"sourceCodeEnd":366,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/auth/KubernetesSecretsTokenAuthProvider.java#L330-L366","documentation":"upsertSecret builds kubernetes actions to create/update the function's authentication secret and waits on a success flag; if the upsert actions don't succeed it throws RuntimeException 'Failed to upsert authentication secret for function <tenant>/<namespace>/<name>'. This blocks updateAuthData from persisting function auth material.","triggerScenarios":"Kubernetes API failure during secret create/update (RBAC lacking create/update on secrets, API unreachable, invalid secret payload e.g. token too large) within updateAuthData.","commonSituations":"Service account missing update/create RBAC on secrets; oversized auth data exceeding etcd's 1MB secret limit; API server connectivity issues from function worker pods.","solutions":["Grant the function worker RBAC create/update/get on secrets in the target namespace","Check the Kubernetes API server reachability and auth from the worker pod","Inspect secret size; reduce function auth data if it exceeds the ~1MB limit","Check worker logs/events (kubectl describe pod, kubectl get events) for the underlying API error and retry"],"exampleFix":"// before\n# worker service account has only 'get' on secrets\n// after\nkubectl create role secret-writer --verb=create,update,get --resource=secrets -n <ns>\nkubectl create rolebinding fn-worker-secret-writer --role=secret-writer \\\n  --serviceaccount=<ns>:function-worker -n <ns>","handlingStrategy":"retry","validationCode":"// preflight\n// kubectl auth can-i create secrets --as=system:serviceaccount:<ns>:function-worker -n <ns>\n// and verify function auth data size < ~1MB","typeGuard":null,"tryCatchPattern":"try {\n    provider.updateAuthData(funcDetails, existing, secretsProvider);\n} catch (RuntimeException e) {\n    if (e.getMessage().startsWith(\"Failed to upsert authentication secret\")) {\n        // inspect underlying k8s error in logs; fix RBAC/size; retry\n    }\n}","preventionTips":["Grant create/update on secrets to the function worker SA","Keep function auth data small (etcd ~1MB secret limit)","Monitor k8s API health from worker pods"],"tags":["kubernetes","secrets","rbac","authentication"],"backgroundTag":"kubernetes-secret-operation-failed","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}