{"record":{"id":"e8c4a20a8506bded","repo":"apache/druid","slug":"expection-in-watching-pods-code-d-and-error-s","errorCode":null,"errorMessage":"Expection in watching pods, code[%d] and error[%s].","messagePattern":"Expection in watching pods, code\\[(.+?)\\] and error\\[(.+?)\\]\\.","errorType":"exception","errorClass":"RuntimeException (Druid RE)","httpStatus":null,"severity":"error","filePath":"extensions-core/kubernetes-extensions/src/main/java/org/apache/druid/k8s/discovery/DefaultK8sApiClient.java","lineNumber":309,"sourceCode":"        @Override\n        public void close()\n        {\n          try {\n            watch.close();\n          }\n          catch (IOException ex) {\n            throw new RE(ex, \"Exception while closing watch.\");\n          }\n        }\n      };\n    }\n    catch (ApiException ex) {\n      if (ex.getCode() == 410) {\n        // k8s no longer has history that we need\n        return null;\n      }\n\n      throw new RE(ex, \"Expection in watching pods, code[%d] and error[%s].\", ex.getCode(), ex.getResponseBody());\n    }\n  }\n}\n","sourceCodeStart":291,"sourceCodeEnd":313,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/kubernetes-extensions/src/main/java/org/apache/druid/k8s/discovery/DefaultK8sApiClient.java#L291-L313","documentation":"Wraps a Kubernetes ApiException from the watch-pods call when the failure is not 410 Gone. The message includes the API response code and raw response body. 410 is deliberately handled as 'resource version history gone' and returns null so the caller can resync; any other failure (401/403/5xx, connection issues) raises this RE.","triggerScenarios":"DefaultK8sApiClient.watchPods receives a non-410 ApiException from the Kubernetes API while establishing or streaming the pod watch — e.g. expired token, RBAC denial, API server 500, or connection refused.","commonSituations":"Service account lacking list/watch RBAC on pods (403); expired or misconfigured kube credentials (401); K8s API server outage (5xx); wrong namespace or label selector.","solutions":["Read code and error body in the message: 403 → fix RBAC role (get/list/watch pods in the namespace); 401 → fix service-account token.","Confirm the namespace and labelSelector in druid.k8s.* config point at real Druid pods.","Check K8s API server health (kubectl get pods) and network path from the Druid process.","After fixing, restart/retry; discovery resyncs from a fresh watch."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// pre-flight: ensure RBAC and namespace before watching\nboolean canWatch = authCanI(\"list\", \"pods\", namespace); // kubectl auth can-i list pods -n <ns>\nif (!canWatch) throw new IllegalStateException(\"Service account cannot watch pods in \" + namespace);","typeGuard":null,"tryCatchPattern":"try {\n  WatchResult wr = client.watchPods(ns, selector, rv, role);\n  if (wr == null) { /* 410: resync from list */ }\n} catch (RE e) {\n  Throwable c = e.getCause();\n  if (c instanceof ApiException && ((ApiException) c).getCode() == 403) { fixRbac(); }\n  else { backoffAndRetry(); }\n}","preventionTips":["Grant list/watch on pods in the discovery namespace via RBAC.","Validate namespace and label selector config at startup.","Handle 410 (null return) by listing pods and restarting the watch from the new resource version.","Add exponential backoff for transient 5xx/network failures."],"tags":["kubernetes","watch","apiexception","rbac","network"],"backgroundTag":"http-error-response","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}