{"record":{"id":"4719b152fcbbfa14","repo":"argoproj/argo-workflows","slug":"failed-to-get-webhook-clients-w","errorCode":null,"errorMessage":"failed to get webhook clients: %w","messagePattern":"failed to get webhook clients: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"server/auth/webhook/interceptor.go","lineNumber":75,"sourceCode":"\t}\n}\n\nfunc (i *Interceptor) addWebhookAuthorization(r *http.Request, kube kubernetes.Interface) error {\n\t// try and exit quickly before we do anything API calls\n\tif r.Method != http.MethodPost || len(r.Header[\"Authorization\"]) > 0 || !strings.HasPrefix(r.URL.Path, pathPrefix) {\n\t\treturn nil\n\t}\n\tparts := strings.SplitN(strings.TrimPrefix(r.URL.Path, pathPrefix), \"/\", 2)\n\tif len(parts) != 2 {\n\t\treturn nil\n\t}\n\tnamespace := parts[0]\n\tsecretsInterface := kube.CoreV1().Secrets(namespace)\n\tctx := r.Context()\n\n\twebhookClients, err := secretsInterface.Get(ctx, \"argo-workflows-webhook-clients\", metav1.GetOptions{})\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to get webhook clients: %w\", err)\n\t}\n\t// we need to read the request body to check the signature, but we still need it for the GRPC request,\n\t// so read it all now, and then reinstate when we are done.\n\t// Limit to 2MB to prevent denial-of-service via oversized webhook payloads.\n\tconst maxWebhookSize = 2 * 1024 * 1024 // 2MB\n\tbuf, err2 := io.ReadAll(io.LimitReader(r.Body, maxWebhookSize+1))\n\tif err2 != nil {\n\t\treturn fmt.Errorf(\"failed to read webhook request body: %w\", err2)\n\t}\n\tif len(buf) > maxWebhookSize {\n\t\treturn fmt.Errorf(\"webhook request body exceeds maximum size of 2MB\")\n\t}\n\tdefer func() { r.Body = io.NopCloser(bytes.NewBuffer(buf)) }()\n\tserviceAccountInterface := kube.CoreV1().ServiceAccounts(namespace)\n\tfor serviceAccountName, data := range webhookClients.Data {\n\t\tr.Body = io.NopCloser(bytes.NewBuffer(buf))\n\t\tclient := &webhookClient{}\n\t\terr := yaml.Unmarshal(data, client)","sourceCodeStart":57,"sourceCodeEnd":93,"githubUrl":"https://github.com/argoproj/argo-workflows/blob/35bff19146f5a6ada77468c431f2624bd577e373/server/auth/webhook/interceptor.go#L57-L93","documentation":"The webhook auth interceptor fetches the `argo-workflows-webhook-clients` Secret in the request's namespace to look up HMAC credentials; any failure reading it (NotFound, RBAC denial, API server error) is wrapped in this error and the webhook request is rejected.","triggerScenarios":"An HTTP request hits a webhook endpoint path and the argo-server's service account cannot GET the Secret `argo-workflows-webhook-clients` in the target namespace — secret not created, wrong namespace in the URL, or RBAC not granting secrets get.","commonSituations":"Upgrading to a version that requires the webhook secret without creating it; webhook URL namespace typo; ClusterRole/Role lacking `secrets` `get` for the argo-server service account; Kubernetes API outage.","solutions":["Create the Secret `argo-workflows-webhook-clients` in the target namespace with the client credentials","Verify the webhook URL namespace segment is correct","Grant the argo-server service account RBAC get on secrets in that namespace","Check k8s API health if errors are cluster-wide"],"exampleFix":"kubectl create secret generic argo-workflows-webhook-clients \\\n  -n <namespace> \\\n  --from-literal=bitbucket=shh... \\\n  --from-literal=github=shh...","handlingStrategy":"validation","validationCode":"_, err := kube.CoreV1().Secrets(ns).Get(ctx, \"argo-workflows-webhook-clients\", metav1.GetOptions{})\nif apierrors.IsNotFound(err) { /* create secret or skip webhook auth */ }","typeGuard":null,"tryCatchPattern":"if err := webhook.Verify(r); err != nil {\n    if apierrors.IsNotFound(errors.Unwrap(err)) {\n        // secret missing: provision it or return 503 with setup hint\n    }\n    return status.Error(codes.Unauthenticated, \"webhook auth failed\")\n}","preventionTips":["Deploy the webhook-clients secret via Helm/kustomize in every namespace used","Verify RBAC for the argo-server SA grants secrets get","Monitor 401/5xx on webhook routes and alert","Validate namespace in webhook URLs after upgrades"],"tags":["webhook","kubernetes","rbac","secret"],"backgroundTag":"secret-not-found","analyzedSha":"35bff19146f5a6ada77468c431f2624bd577e373","analyzedAt":"2026-09-03T19:34:35.908Z","contentChangedAt":"2026-09-03T19:34:35.908Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}