apache/hadoop · error · RpcException

RPC response has invalid length of %d

Error message

RPC response has invalid length of %d

What it means

IpcStreams.readResponse reads a 4-byte framed RPC response; after the first-response -1 handshake-error case is handled, any length <= 0 throws RpcException('RPC response has invalid length of N'). A non-positive frame length is impossible in the protocol, so the byte stream is no longer aligned with the framing — most commonly a client/server Hadoop version or wire-protocol mismatch, or a corrupted/garbage stream (e.g., an HTTP proxy or wrong service answering on the port).

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java:1959

    private void setOutputStream(OutputStream os) {
      this.out = (os instanceof DataOutputStream)
          ? (DataOutputStream)os : new DataOutputStream(os);
    }

    public ByteBuffer readResponse() throws IOException {
      int length = in.readInt();
      if (firstResponse) {
        firstResponse = false;
        // pre-rpcv9 exception, almost certainly a version mismatch.
        if (length == -1) {
          in.readInt(); // ignore fatal/error status, it's fatal for us.
          throw new RemoteException(WritableUtils.readString(in),
                                    WritableUtils.readString(in));
        }
      }
      if (length <= 0) {
        throw new RpcException(String.format("RPC response has " +
            "invalid length of %d", length));
      }
      if (maxResponseLength > 0 && length > maxResponseLength) {
        throw new RpcException(String.format("RPC response has a " +
            "length of %d exceeds maximum data length", length));
      }
      ByteBuffer bb = ByteBuffer.allocate(length);
      in.readFully(bb.array());
      return bb;
    }

    public void sendRequest(byte[] buf) throws IOException {
      out.write(buf);
    }

    @Override
    public void flush() throws IOException {
      out.flush();

View on GitHub (pinned to 2add963021)

Solutions

1. Verify the endpoint: confirm the address/port is a real Hadoop RPC server (e.g., dfs.namenode.rpc-address, not the http one).
2. Align client and server Hadoop versions/protocol implementations (same artifacts for both sides of any custom protocol).
3. Close the proxy and reconnect with a fresh connection so framing resynchronizes; if it recurs, capture the first bytes received to identify what is actually answering.

Example fix

// before
Configuration conf = new Configuration();
conf.set("fs.defaultFS", "hdfs://nn:9870"); // 9870 is the HTTP port
FileSystem fs = FileSystem.get(conf); // RPC framing broken -> invalid length

// after
Configuration conf = new Configuration();
conf.set("fs.defaultFS", "hdfs://nn:8020"); // the RPC port (dfs.namenode.rpc-address)
FileSystem fs = FileSystem.get(conf);

try {
  return proxy.call(req);
} catch (RpcException e) {
  if (e.getMessage() != null && e.getMessage().contains("invalid length")) {
    // framing desync: discard the proxy/connection entirely
    RPC.stopProxy(proxy);
    proxy = buildFreshProxy();
    throw new IllegalStateException("RPC wire mismatch - verify endpoint and versions", e);
  }
  throw e;
}

When it happens

Trigger: A Hadoop client whose IPC wire version differs from the server's answers with a non-framed payload; connecting an RPC client to a port served by a different protocol (HTTP server, HTTPS endpoint); a truncated/corrupted TCP stream after a mid-read network fault being read as the next frame.

Common situations: Mixing Hadoop versions across a cluster upgrade (e.g., 2.x client vs 3.x server on a non-compatible protocol); connecting to a webhdfs/HTTP port with an RPC client; JVM/serializer divergence on custom Writables; misconfigured port in fs.defaultFS or yarn.resourcemanager.address.

Related errors

• Call interrupted
• Failed to get connection for {}, {}: {} is already stopped
• RPC response has a length of %d exceeds maximum data length
• Too many or few parameters for request. Method: [{}], Expect
• null param while calling Method: [{}]

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/880c15eb9506269f. Report an issue: GitHub.