apache/hadoop · error · IOException

Unknown op {} in data stream

Error message

Unknown op {} in data stream

What it means

Receiver.processOp switches over the op code parsed off the data-transfer stream; any op value outside the handled set (READ_BLOCK, WRITE_BLOCK, block transfer, short-circuit ops, etc.) hits the default branch and throws IOException 'Unknown op N in data stream'.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java:137

      opBlockChecksum(in);
      break;
    case BLOCK_GROUP_CHECKSUM:
      opStripedBlockChecksum(in);
      break;
    case TRANSFER_BLOCK:
      opTransferBlock(in);
      break;
    case REQUEST_SHORT_CIRCUIT_FDS:
      opRequestShortCircuitFds(in);
      break;
    case RELEASE_SHORT_CIRCUIT_FDS:
      opReleaseShortCircuitFds(in);
      break;
    case REQUEST_SHORT_CIRCUIT_SHM:
      opRequestShortCircuitShm(in);
      break;
    default:
      throw new IOException("Unknown op " + op + " in data stream");
    }
  }

  static private CachingStrategy getCachingStrategy(CachingStrategyProto strategy) {
    Boolean dropBehind = strategy.hasDropBehind() ?
        strategy.getDropBehind() : null;
    Long readahead = strategy.hasReadahead() ?
        strategy.getReadahead() : null;
    return new CachingStrategy(dropBehind, readahead);
  }

  /** Receive OP_READ_BLOCK */
  private void opReadBlock() throws IOException {
    OpReadBlockProto proto = OpReadBlockProto.parseFrom(vintPrefixed(in));
    TraceScope traceScope = continueTraceSpan(proto.getHeader(),
        proto.getClass().getSimpleName());
    try {
      readBlock(PBHelperClient.convert(proto.getHeader().getBaseHeader().getBlock()),

View on GitHub (pinned to 2add963021)

Solutions

1. Check for version skew between the sender and this DataNode; upgrade the DN to a version that understands the op
2. If versions match, suspect stream corruption: capture the op value from DN logs and inspect the connection (MTU/TLS/keepalive issues)
3. For custom clients, restrict yourself to ops the target DataTransferProtocol implements

try {
  receiver.processOp(in);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Unknown op")) {
    // unsupported or desynchronized op stream: close the connection; log the op
    // code and peer for diagnosis rather than continuing on the same stream
  } else {
    throw e;
  }
}

When it happens

Trigger: A newer-version peer sends an operation this DataNode does not implement; or the stream desynchronizes (partial read, corrupted framing) so subsequent bytes decode to an invalid op code.

Common situations: Rolling upgrade with a new block operation reaching an old DN; custom clients with framing bugs; connection/stream corruption after network or TLS faults; fuzzed input on the xfer port.

Related errors

• Version Mismatch (Expected: {}, Received: {} )
• Cannot create a secured connection if DataNode listens on un
• Received %x instead of %x from client.
• file VERSION is invalid.
• A disk IO error occurred

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/9281fd1b619b9aa2. Report an issue: GitHub.