{"record":{"id":"6e462ab23f6d78b6","repo":"apache/druid","slug":"failed-to-patch-pod-s-s-code-d-error-s","errorCode":null,"errorMessage":"Failed to patch pod[%s/%s], code[%d], error[%s].","messagePattern":"Failed to patch pod\\[(.+?)/(.+?)\\], code\\[(.+?)\\], error\\[(.+?)\\]\\.","errorType":"exception","errorClass":"RuntimeException (Druid RE)","httpStatus":null,"severity":"error","filePath":"extensions-core/kubernetes-extensions/src/main/java/org/apache/druid/k8s/discovery/DefaultK8sApiClient.java","lineNumber":89,"sourceCode":"      PatchUtils.patch(\n          V1Pod.class,\n          () -> coreV1Api.patchNamespacedPodCall(\n              podName,\n              podNamespace,\n              new V1Patch(jsonPatchStr),\n              \"true\",\n              null,\n              null,\n              null,\n              null,\n              null\n          ),\n          V1Patch.PATCH_FORMAT_JSON_PATCH,\n          realK8sClient\n      );\n    }\n    catch (ApiException ex) {\n      throw new RE(ex, \"Failed to patch pod[%s/%s], code[%d], error[%s].\", podNamespace, podName, ex.getCode(), ex.getResponseBody());\n    }\n  }\n\n  @Override\n  public DiscoveryDruidNodeList listPods(\n      String podNamespace,\n      String labelSelector,\n      NodeRole nodeRole\n  )\n  {\n    try {\n      V1PodList podList = coreV1Api.listNamespacedPod(\n          podNamespace,\n          null,\n          null,\n          null,\n          null,\n          labelSelector,","sourceCodeStart":71,"sourceCodeEnd":107,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/kubernetes-extensions/src/main/java/org/apache/druid/k8s/discovery/DefaultK8sApiClient.java#L71-L107","documentation":"DefaultK8sApiClient.patchPod calls the Kubernetes API to JSON-patch a pod. If the Kubernetes client raises ApiException, it is rethrown as a Druid RE with the pod namespace/name, HTTP code, and response body. This means the patch request failed at the Kubernetes API server.","triggerScenarios":"patchPod(podNamespace, podName, content) where the API server returns an error: 404 pod not found, 409/422 patch conflict or invalid patch body, 401/403 auth failures, or connection errors to the API server.","commonSituations":"Pod already terminated (task finished) so the patch 404s; RBAC not granting 'pods/patch'; malformed JSON patch payload; k8s client kubeconfig/service-account misconfiguration; brief API-server unavailability during node upgrades.","solutions":["Read the code and responseBody in the RE: 404 means the pod is gone (often benign during shutdown); 403 means fix RBAC to allow patch on pods","Validate the JSON patch body against the pod spec (correct op/path/value structure)","Check pod existence before patching, or handle 404 as a no-op","Verify kubeconfig/service account credentials and API server connectivity"],"exampleFix":"// before\nk8sClient.patchPod(ns, name, patch);\n// after\ntry {\n  k8sClient.patchPod(ns, name, patch);\n} catch (RE e) {\n  if (!e.getMessage().contains(\"code[404]\")) throw e;\n  // pod already gone — safe to skip\n}","handlingStrategy":"try-catch","validationCode":"// precheck pod existence and patchability\nV1Pod pod = coreV1Api.readNamespacedPod(podName, podNamespace).execute();\nif (!\"Running\".equals(pod.getStatus().getPhase())) throw new IllegalStateException(\"pod not Running\");","typeGuard":"static boolean isNotFound(ApiException ex) { return ex.getCode() == 404; }","tryCatchPattern":"try {\n  client.patchPod(ns, name, patch);\n} catch (RE e) {\n  if (e.getMessage().contains(\"code[404]\")) {\n    LOG.warn(\"pod %s/%s already gone; skipping patch\", ns, name);\n  } else if (e.getMessage().contains(\"code[403]\")) {\n    throw new IllegalStateException(\"RBAC: grant pods/patch\", e);\n  } else throw e;\n}","preventionTips":["Grant the service account patch permission on pods in the druid namespace","Treat 404 as benign when patching during shutdown — pods may have terminated","Validate JSON patch op/path/value against the pod schema before sending","Add retries with backoff for 5xx/429 from the API server"],"tags":["kubernetes","pod-patch","api-client"],"backgroundTag":"api-error-response","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}