{"record":{"id":"80c270e30299fd41","repo":"apache/hadoop","slug":"unexpected-data-length-datalength-from","errorCode":null,"errorMessage":"\"Unexpected data length \" + dataLength + \"!! from \" + getHostAddress()","messagePattern":"\"Unexpected data length \" \\+ dataLength \\+ \"!! from \" \\+ getHostAddress\\(\\)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java","lineNumber":2518,"sourceCode":"    }\n\n    private void disposeSasl() {\n      if (saslServer != null) {\n        try {\n          saslServer.dispose();\n        } catch (SaslException ignored) {\n        } finally {\n          saslServer = null;\n        }\n      }\n    }\n\n    private void checkDataLength(int dataLength) throws IOException {\n      if (dataLength < 0) {\n        String error = \"Unexpected data length \" + dataLength +\n                       \"!! from \" + getHostAddress();\n        LOG.warn(error);\n        throw new IOException(error);\n      } else if (dataLength > maxDataLength) {\n        String error = \"Requested data length \" + dataLength +\n              \" is longer than maximum configured RPC length \" + \n            maxDataLength + \".  RPC came from \" + getHostAddress();\n        LOG.warn(error);\n        throw new IOException(error);\n      }\n    }\n\n    /**\n     * This method reads in a non-blocking fashion from the channel: \n     * this method is called repeatedly when data is present in the channel; \n     * when it has enough data to process one rpc it processes that rpc.\n     * \n     * On the first pass, it processes the connectionHeader, \n     * connectionContext (an outOfBand RPC) and at most one RPC request that \n     * follows that. On future passes it will process at most one RPC request.\n     *  ","sourceCodeStart":2500,"sourceCodeEnd":2536,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java#L2500-L2536","documentation":"Every Hadoop IPC frame starts with a 4-byte big-endian length. checkDataLength() rejects any frame whose declared length is negative, which cannot happen in a valid RPC stream, so it almost always means the bytes on the socket are not Hadoop IPC at all (wrong protocol, TLS to a plaintext port, or stream corruption). The offending connection is closed after a WARN log.","triggerScenarios":"An HTTPS/TLS client or curl https:// connects to the unencrypted RPC port and the TLS bytes decode to a negative integer; a garbage payload from a port scanner; a client speaking HTTP, Redis, or any other protocol to the RPC listener; memory/socket corruption mid-stream.","commonSituations":"Misconfigured service discovery or load balancer routing TLS traffic to the IPC port; operators testing connectivity with netcat/HTTP clients against the RPC port; middleboxes or proxies rewriting traffic; mixing up RPC and HTTP ports in configuration (e.g. pointing fs.defaultFS at the wrong port).","solutions":["Verify the client connects to the true RPC port (e.g. NameNode rpc-address, not the http-address) and that the host address comes from a verified source","Ensure TLS/SSL is either enabled on both client and server or on neither for that port","Check what is actually connecting: the log line prints getHostAddress() of the offending peer","Remove any load balancer, sidecar, or proxy that forwards non-IPC traffic to the RPC listener"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// before opening a connection, sanity-check the endpoint is an IPC port\nInetSocketAddress rpcAddr = NetUtils.createSocketAddr(\n    conf.get(\"fs.defaultFS\"), 8020, \"fs.defaultFS\");","typeGuard":null,"tryCatchPattern":"try {\n  proxy = RPC.getProxy(protocol, versionID, addr, conf);\n} catch (IOException e) {\n  if (String.valueOf(e.getMessage()).contains(\"Unexpected data length\")) {\n    // peer is not speaking Hadoop IPC on this port (e.g. TLS to plaintext port)\n    LOG.error(\"{} is not a Hadoop RPC endpoint; check scheme/port\", addr, e);\n  }\n  throw e;\n}","preventionTips":["Derive RPC addresses from verified config keys, never hardcode","Keep TLS listeners and plaintext RPC listeners on distinct, documented ports","Exclude RPC ports from load-balancer and health-check pools"],"tags":["rpc","framing","protocol-mismatch","network","hadoop-ipc"],"backgroundTag":"rpc-framing-invalid","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}