{"record":{"id":"0d332bf681058843","repo":"apache/druid","slug":"failed-to-deserialize-discoverydruidnode-s","errorCode":null,"errorMessage":"Failed to deserialize DiscoveryDruidNode[%s]","messagePattern":"Failed to deserialize DiscoveryDruidNode\\[(.+?)\\]","errorType":"exception","errorClass":"RuntimeException (Druid RE)","httpStatus":null,"severity":"error","filePath":"extensions-core/kubernetes-extensions/src/main/java/org/apache/druid/k8s/discovery/DefaultK8sApiClient.java","lineNumber":161,"sourceCode":"  {\n    if (pod.getStatus() == null) {\n      return false;\n    }\n    List<V1ContainerStatus> containerStatuses = pod.getStatus().getContainerStatuses();\n    if (containerStatuses == null || containerStatuses.isEmpty()) {\n      return false;\n    }\n    return containerStatuses.stream().allMatch(cs -> Boolean.TRUE.equals(cs.getReady()));\n  }\n\n  private DiscoveryDruidNode getDiscoveryDruidNodeFromPodDef(NodeRole nodeRole, V1Pod podDef)\n  {\n    String jsonStr = podDef.getMetadata().getAnnotations().get(K8sDruidNodeAnnouncer.getInfoAnnotation(nodeRole));\n    try {\n      return jsonMapper.readValue(jsonStr, DiscoveryDruidNode.class);\n    }\n    catch (JsonProcessingException ex) {\n      throw new RE(ex, \"Failed to deserialize DiscoveryDruidNode[%s]\", jsonStr);\n    }\n  }\n\n  @Override\n  public WatchResult watchPods(String namespace, String labelSelector, String lastKnownResourceVersion, NodeRole nodeRole)\n  {\n    try {\n      Watch<V1Pod> watch =\n          Watch.createWatch(\n              realK8sClient,\n              coreV1Api.listNamespacedPodCall(\n                  namespace,\n                  null,\n                  true,\n                  null,\n                  null,\n                  labelSelector,\n                  null,","sourceCodeStart":143,"sourceCodeEnd":179,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/kubernetes-extensions/src/main/java/org/apache/druid/k8s/discovery/DefaultK8sApiClient.java#L143-L179","documentation":"Thrown by DefaultK8sApiClient when the JSON stored in a Druid pod's Kubernetes annotation cannot be deserialized into a DiscoveryDruidNode via Jackson. The k8s driver-based discovery reads node info from the annotation named by K8sDruidNodeAnnouncer.getInfoAnnotation(nodeRole); if that payload is malformed, missing required fields, or was written by an incompatible Druid version, Jackson throws JsonProcessingException which is wrapped in this RE with the offending JSON string.","triggerScenarios":"Calling getDiscoveryDruidNodeFromPodDef for a pod whose info annotation contains invalid JSON, null (annotation absent), or a JSON payload that does not match DiscoveryDruidNode's schema (wrong fields/types). Raised from getDiscoveryDruidNodeFromPodDef, reached via node() and hasNext() while iterating discovered nodes.","commonSituations":"Mixed Druid cluster versions where an older node announces an annotation schema the reader can't parse; manual edits or truncated annotations; druid_k8s annotation keys changed between releases; pods from other frameworks accidentally carrying the label selector.","solutions":["Inspect the JSON string in the error message; validate it parses and matches DiscoveryDruidNode (druid_node role, host, port, etc.).","Verify all pods in the watched namespace/labelSelector run a compatible Druid version announcing the same annotation format.","Check the annotation key for the nodeRole matches what K8sDruidNodeAnnouncer writes (getInfoAnnotation(nodeRole)).","Delete/re-announce stale pods or fix the annotation contents, then retry discovery."],"exampleFix":"// before: annotation written manually with wrong shape\nkubectl annotate pod druid-historicals-0 druid.information.historical='{\"host\":\"h\",\"port\":8083}'\n// after: let Druid announce it, or match DiscoveryDruidNode schema incl. druidNode and serverDiscParams\nkubectl annotate pod druid-historicals-0 druid.information.historical='{\"druidNode\":{\"host\":\"h\",\"port\":8083,\"plaintextPort\":8083},\"nodeType\":\"historical\",\"services\":{...}}'","handlingStrategy":"validation","validationCode":"String ann = pod.getMetadata() == null || pod.getMetadata().getAnnotations() == null ? null : pod.getMetadata().getAnnotations().get(\"druid.information.\" + nodeRole);\nif (ann == null) throw new IllegalStateException(\"Missing druid info annotation for role \" + nodeRole);\ntry { new ObjectMapper().readTree(ann); } catch (Exception e) { throw new IllegalStateException(\"Invalid JSON in annotation: \" + e.getMessage()); }","typeGuard":"boolean hasValidNodeAnnotation(Pod p, String role) {\n  Map<String,String> anns = p.getMetadata() != null ? p.getMetadata().getAnnotations() : null;\n  String json = anns == null ? null : anns.get(\"druid.information.\" + role);\n  if (json == null) return false;\n  try { JSON_MAPPER.readValue(json, DiscoveryDruidNode.class); return true; } catch (Exception e) { return false; }\n}","tryCatchPattern":"try {\n  DiscoveryDruidNode node = jsonMapper.readValue(jsonStr, DiscoveryDruidNode.class);\n} catch (JsonProcessingException ex) {\n  LOG.warn(ex, \"Skipping pod with undecodable node annotation: %.100s\", jsonStr);\n  return null; // skip rather than fail the whole discovery iteration\n}","preventionTips":["Pin all Druid pods to the same version so annotation schemas match.","Never hand-edit druid.information.* annotations; let the announcer write them.","Verify pod label selectors exclude non-Druid pods.","Log the annotation JSON on parse failure for fast diagnosis."],"tags":["kubernetes","jackson","deserialization","discovery"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}