{"record":{"id":"5aaa9e937f478495","repo":"apache/hadoop","slug":"unexpected-format-of-jmx-json-response-for","errorCode":null,"errorMessage":"Unexpected format of JMX JSON response for: {}","messagePattern":"Unexpected format of JMX JSON response for: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/java/org/apache/hadoop/tools/dynamometer/DynoInfraUtils.java","lineNumber":564,"sourceCode":"          nnWebUri.getPort(), \"/jmx?qry=\" + jmxBeanQuery);\n    } catch (MalformedURLException e) {\n      throw new IllegalArgumentException(\"Invalid JMX query: \\\"\" + jmxBeanQuery\n          + \"\\\" against \" + \"NameNode URI: \" + nnWebUri);\n    }\n    HttpURLConnection conn = (HttpURLConnection) queryURL.openConnection();\n    if (conn.getResponseCode() != 200) {\n      throw new IOException(\n          \"Unable to retrieve JMX: \" + conn.getResponseMessage());\n    }\n    InputStream in = conn.getInputStream();\n    JsonFactory fac = new JsonFactory();\n    JsonParser parser = fac.createParser(in);\n    if (parser.nextToken() != JsonToken.START_OBJECT\n        || parser.nextToken() != JsonToken.FIELD_NAME\n        || !parser.getCurrentName().equals(\"beans\")\n        || parser.nextToken() != JsonToken.START_ARRAY\n        || parser.nextToken() != JsonToken.START_OBJECT) {\n      throw new IOException(\n          \"Unexpected format of JMX JSON response for: \" + jmxBeanQuery);\n    }\n    int objectDepth = 1;\n    String ret = null;\n    while (objectDepth > 0) {\n      JsonToken tok = parser.nextToken();\n      if (tok == JsonToken.START_OBJECT) {\n        objectDepth++;\n      } else if (tok == JsonToken.END_OBJECT) {\n        objectDepth--;\n      } else if (tok == JsonToken.FIELD_NAME) {\n        if (parser.getCurrentName().equals(property)) {\n          parser.nextToken();\n          ret = parser.getText();\n          break;\n        }\n      }\n    }","sourceCodeStart":546,"sourceCodeEnd":582,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/java/org/apache/hadoop/tools/dynamometer/DynoInfraUtils.java#L546-L582","documentation":"DynoInfraUtils polls a Hadoop daemon's JMX servlet (e.g. http://<host>:<httpPort>/jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo) with an HTTP GET and parses the body as a strict Jackson token stream that must begin with {\"beans\":[{ . If the HTTP status is 200 but the token sequence does not match START_OBJECT -> FIELD_NAME 'beans' -> START_ARRAY -> START_OBJECT, this IOException is thrown. It means the endpoint answered, but the payload is not a JMX beans document.","triggerScenarios":"A 200 response whose body is not the expected JMX shape: the URL points at the RPC port or a UI page instead of the JMX servlet, an authentication filter or proxy returns a 200 HTML login/error page, or the qry= bean filter matches zero beans so the document is {\"beans\":[]}.","commonSituations":"Using the NameNode RPC port (8020/9000) instead of the HTTP port (9870/50070), a gateway/SSL proxy in front of the daemon rewriting responses, polling during daemon startup before JMX beans register, or a typo in the qry= MBean name.","solutions":["curl the exact jmxBeanQuery URL shown in the message and confirm the body starts with {\"beans\":[{","Point the query at the daemon's HTTP server port (dfs.namenode.http-address / dfs.datanode.http-address), not the RPC port","Check that the qry= value matches the MBean exactly, e.g. Hadoop:service=NameNode,name=NameNodeInfo","If polling during startup, retry with backoff until the bean is registered instead of failing immediately"],"exampleFix":"// before: JMX URL pointed at the NameNode RPC port\nString url = \"http://nn1:8020/jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo\";\n// after: use the HTTP server port and verify shape first\nString url = \"http://nn1:9870/jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo\";\n// curl \"$url\" must print a body starting with {\"beans\":[{","handlingStrategy":"validation","validationCode":"// pre-flight: confirm the endpoint returns a beans document before polling\ntry (InputStream in = new URL(jmxUrl).openStream()) {\n  JsonParser p = new JsonFactory().createParser(in);\n  boolean ok = p.nextToken() == JsonToken.START_OBJECT\n      && p.nextToken() == JsonToken.FIELD_NAME && \"beans\".equals(p.getCurrentName())\n      && p.nextToken() == JsonToken.START_ARRAY\n      && p.nextToken() == JsonToken.START_OBJECT;\n  if (!ok) throw new IllegalStateException(\"endpoint does not serve JMX beans: \" + jmxUrl);\n}","typeGuard":null,"tryCatchPattern":"catch (IOException e) { /* log jmxBeanQuery; sleep; retry with backoff until startup timeout, since a daemon mid-boot may briefly serve odd 200 payloads */ }","preventionTips":["Always curl the exact jmxBeanQuery URL once before wiring it into scripts","Keep daemon HTTP ports in variables distinct from RPC ports to avoid mixups","Treat a 200-with-HTML the same as a failure: validate the first JSON tokens, not just the status code"],"tags":["dynamometer","jmx","json","http","startup-check"],"backgroundTag":"invalid-json-response-format","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}