{"record":{"id":"ccc1b5795d86e3b5","repo":"argoproj/argo-workflows","slug":"failed-to-read-webhook-request-body-w","errorCode":null,"errorMessage":"failed to read webhook request body: %w","messagePattern":"failed to read webhook request body: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"server/auth/webhook/interceptor.go","lineNumber":83,"sourceCode":"\tparts := strings.SplitN(strings.TrimPrefix(r.URL.Path, pathPrefix), \"/\", 2)\n\tif len(parts) != 2 {\n\t\treturn nil\n\t}\n\tnamespace := parts[0]\n\tsecretsInterface := kube.CoreV1().Secrets(namespace)\n\tctx := r.Context()\n\n\twebhookClients, err := secretsInterface.Get(ctx, \"argo-workflows-webhook-clients\", metav1.GetOptions{})\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to get webhook clients: %w\", err)\n\t}\n\t// we need to read the request body to check the signature, but we still need it for the GRPC request,\n\t// so read it all now, and then reinstate when we are done.\n\t// Limit to 2MB to prevent denial-of-service via oversized webhook payloads.\n\tconst maxWebhookSize = 2 * 1024 * 1024 // 2MB\n\tbuf, err2 := io.ReadAll(io.LimitReader(r.Body, maxWebhookSize+1))\n\tif err2 != nil {\n\t\treturn fmt.Errorf(\"failed to read webhook request body: %w\", err2)\n\t}\n\tif len(buf) > maxWebhookSize {\n\t\treturn fmt.Errorf(\"webhook request body exceeds maximum size of 2MB\")\n\t}\n\tdefer func() { r.Body = io.NopCloser(bytes.NewBuffer(buf)) }()\n\tserviceAccountInterface := kube.CoreV1().ServiceAccounts(namespace)\n\tfor serviceAccountName, data := range webhookClients.Data {\n\t\tr.Body = io.NopCloser(bytes.NewBuffer(buf))\n\t\tclient := &webhookClient{}\n\t\terr := yaml.Unmarshal(data, client)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to unmarshal webhook client \\\"%s\\\": %w\", serviceAccountName, err)\n\t\t}\n\t\ti.logger.WithFields(logging.Fields{\"serviceAccountName\": serviceAccountName, \"webhookType\": client.Type}).Debug(r.Context(), \"Attempting to match webhook request\")\n\t\tok := webhookParsers[client.Type](client.Secret, r)\n\t\tif ok {\n\t\t\ti.logger.WithField(\"serviceAccountName\", serviceAccountName).Debug(r.Context(), \"Matched webhook request\")\n\t\t\tserviceAccount, err := serviceAccountInterface.Get(ctx, serviceAccountName, metav1.GetOptions{})","sourceCodeStart":65,"sourceCodeEnd":101,"githubUrl":"https://github.com/argoproj/argo-workflows/blob/35bff19146f5a6ada77468c431f2624bd577e373/server/auth/webhook/interceptor.go#L65-L101","documentation":"The interceptor must buffer the entire webhook request body to verify HMAC signatures and then reinstate it for the gRPC handler. If io.ReadAll fails (client disconnect, network error mid-body, timeout) the request is rejected with this wrapped error.","triggerScenarios":"addWebhookAuthorization reads the request body and the underlying connection errors: client aborted the POST, proxy dropped the connection, TLS/network interruption, or request context canceled before the body completed.","commonSituations":"Flaky networks between Git host (GitHub/Bitbucket) and argo-server; aggressive ingress/LLM proxies with short idle timeouts; clients canceling large webhook deliveries.","solutions":["Retry the webhook delivery from the source (GitHub/Bitbucket redeliver)","Check ingress/proxy timeouts and raise them for the argo-server webhook route","Verify TLS configuration between the sender and argo-server isn't causing mid-stream resets","Look at argo-server logs for repeated cancellations to identify the network hop failing"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"if r.ContentLength > maxWebhookSize { /* reject early instead of failing mid-read */ }","typeGuard":null,"tryCatchPattern":"buf, err := io.ReadAll(io.LimitReader(r.Body, maxWebhookSize+1))\nif err != nil {\n    if errors.Is(err, context.Canceled) || errors.Is(err, io.ErrUnexpectedEOF) {\n        // client aborted: log at debug, let provider redeliver\n    }\n    return err\n}","preventionTips":["Raise ingress timeouts for webhook routes","Ask Git providers to redeliver failed webhooks (they auto-retry)","Monitor for mid-body disconnects indicating network issues","Keep payload sizes small on the sender side"],"tags":["webhook","network","io"],"backgroundTag":"request-body-read-failed","analyzedSha":"35bff19146f5a6ada77468c431f2624bd577e373","analyzedAt":"2026-09-03T19:34:35.908Z","contentChangedAt":"2026-09-03T19:34:35.908Z","schemaVersion":2},"datasetVersion":"2026-09-08T15:18:49.778Z"}