{"record":{"id":"7353b90d1d4e72c6","repo":"openfaas/faas","slug":"string-upstreambody","errorCode":null,"errorMessage":"string(upstreamBody)","messagePattern":"string\\(upstreamBody\\)","errorType":"http","errorClass":null,"httpStatus":null,"severity":"error","filePath":"gateway/metrics/add_metrics.go","lineNumber":37,"sourceCode":"\treturn func(w http.ResponseWriter, r *http.Request) {\n\n\t\trecorder := httptest.NewRecorder()\n\t\thandler.ServeHTTP(recorder, r)\n\t\tupstreamCall := recorder.Result()\n\n\t\tif upstreamCall.Body == nil {\n\t\t\tlog.Println(\"Upstream call had empty body.\")\n\t\t\treturn\n\t\t}\n\n\t\tdefer upstreamCall.Body.Close()\n\t\tupstreamBody, _ := io.ReadAll(upstreamCall.Body)\n\n\t\tif recorder.Code != http.StatusOK {\n\t\t\tlog.Printf(\"List functions responded with code %d, body: %s\",\n\t\t\t\trecorder.Code,\n\t\t\t\tstring(upstreamBody))\n\t\t\thttp.Error(w, string(upstreamBody), recorder.Code)\n\t\t\treturn\n\t\t}\n\n\t\tvar functions []types.FunctionStatus\n\n\t\terr := json.Unmarshal(upstreamBody, &functions)\n\t\tif err != nil {\n\t\t\tlog.Printf(\"Metrics upstream error: %s, value: %s\", err, string(upstreamBody))\n\n\t\t\thttp.Error(w, \"Unable to parse list of functions from provider\", http.StatusInternalServerError)\n\t\t\treturn\n\t\t}\n\n\t\t// Ensure values are empty first.\n\t\tfor i := range functions {\n\t\t\tfunctions[i].InvocationCount = 0\n\t\t}\n","sourceCodeStart":19,"sourceCodeEnd":55,"githubUrl":"https://github.com/openfaas/faas/blob/8d803bf9e2655aec6a60fd0e25b392839f96af95/gateway/metrics/add_metrics.go#L19-L55","documentation":"AddMetricsHandler wraps the internal list-functions handler (it serves GET /system/functions when a prometheus_query is configured) and runs it in-process with an httptest.Recorder before enriching the result with invocation metrics. If the wrapped handler returns any status other than 200, the gateway forwards the upstream body and status code verbatim — the message you see is the provider's own error payload, not a fixed gateway string.","triggerScenarios":"GET /system/functions while the provider fails to list functions: faas-netes returning 500 on a Kubernetes API error, 403 from missing RBAC permissions on the gateway's service account, or faasd failing to read its container backend.","commonSituations":"Gateway service account losing list-deployments permissions after a chart upgrade; Kubernetes API unreachable or rate-limiting; provider crash-looping; stale faasd credentials.","solutions":["Read the forwarded body — it contains the provider's error message, often the Kubernetes error text","Check the provider's own logs (kubectl logs on the faas-netes deployment, or the faasd journal)","If the body mentions RBAC/forbidden, fix the gateway service account roles","Curl the provider's /system/functions directly to confirm the failure exists outside the metrics wrapper","If the Kubernetes API was flapping, retry once it recovers"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"resp, err := client.Get(gateway + \"/system/functions\")\nif err != nil || resp.StatusCode >= 500 {\n    // provider-side listing failure: read the forwarded provider error body,\n    // back off and retry; fix RBAC/config if the body says forbidden\n    time.Sleep(backoff)\n    continue\n}","preventionTips":["Alert on the gateway log line 'List functions responded with code'","Re-verify the gateway service account's RBAC after every chart upgrade","Read the response body first — it is the provider's own error message"],"tags":["http","provider","kubernetes","rbac","functions-list"],"backgroundTag":null,"analyzedSha":"8d803bf9e2655aec6a60fd0e25b392839f96af95","analyzedAt":"2026-08-16T00:12:29.759Z","schemaVersion":2},"datasetVersion":"2026-08-16T03:17:38.424Z"}