apache/druid · error · RuntimeException (Druid RE)

Failed to patch pod[%s/%s], code[%d], error[%s].

Error message

Failed to patch pod[%s/%s], code[%d], error[%s].

What it means

DefaultK8sApiClient.patchPod calls the Kubernetes API to JSON-patch a pod. If the Kubernetes client raises ApiException, it is rethrown as a Druid RE with the pod namespace/name, HTTP code, and response body. This means the patch request failed at the Kubernetes API server.

Source

Thrown at extensions-core/kubernetes-extensions/src/main/java/org/apache/druid/k8s/discovery/DefaultK8sApiClient.java:89

      PatchUtils.patch(
          V1Pod.class,
          () -> coreV1Api.patchNamespacedPodCall(
              podName,
              podNamespace,
              new V1Patch(jsonPatchStr),
              "true",
              null,
              null,
              null,
              null,
              null
          ),
          V1Patch.PATCH_FORMAT_JSON_PATCH,
          realK8sClient
      );
    }
    catch (ApiException ex) {
      throw new RE(ex, "Failed to patch pod[%s/%s], code[%d], error[%s].", podNamespace, podName, ex.getCode(), ex.getResponseBody());
    }
  }

  @Override
  public DiscoveryDruidNodeList listPods(
      String podNamespace,
      String labelSelector,
      NodeRole nodeRole
  )
  {
    try {
      V1PodList podList = coreV1Api.listNamespacedPod(
          podNamespace,
          null,
          null,
          null,
          null,
          labelSelector,

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Read the code and responseBody in the RE: 404 means the pod is gone (often benign during shutdown); 403 means fix RBAC to allow patch on pods
2. Validate the JSON patch body against the pod spec (correct op/path/value structure)
3. Check pod existence before patching, or handle 404 as a no-op
4. Verify kubeconfig/service account credentials and API server connectivity

Example fix

// before
k8sClient.patchPod(ns, name, patch);
// after
try {
  k8sClient.patchPod(ns, name, patch);
} catch (RE e) {
  if (!e.getMessage().contains("code[404]")) throw e;
  // pod already gone — safe to skip
}

// precheck pod existence and patchability
V1Pod pod = coreV1Api.readNamespacedPod(podName, podNamespace).execute();
if (!"Running".equals(pod.getStatus().getPhase())) throw new IllegalStateException("pod not Running");

static boolean isNotFound(ApiException ex) { return ex.getCode() == 404; }

try {
  client.patchPod(ns, name, patch);
} catch (RE e) {
  if (e.getMessage().contains("code[404]")) {
    LOG.warn("pod %s/%s already gone; skipping patch", ns, name);
  } else if (e.getMessage().contains("code[403]")) {
    throw new IllegalStateException("RBAC: grant pods/patch", e);
  } else throw e;
}

When it happens

Trigger: patchPod(podNamespace, podName, content) where the API server returns an error: 404 pod not found, 409/422 patch conflict or invalid patch body, 401/403 auth failures, or connection errors to the API server.

Common situations: Pod already terminated (task finished) so the patch 404s; RBAC not granting 'pods/patch'; malformed JSON patch payload; k8s client kubeconfig/service-account misconfiguration; brief API-server unavailability during node upgrades.

Related errors

• Expection in listing pods, code[%d] and error[%s].
• Failed to deserialize DiscoveryDruidNode[%s]
• Exception while closing watch.
• Expection in watching pods, code[%d] and error[%s].
• Failed to get current leader for [%s]

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/6e462ab23f6d78b6. Report an issue: GitHub.