apache/hadoop · error · InvalidMagicNumberException

Received %x instead of %x from client.

Error message

Received %x instead of %x from client.

What it means

SaslDataTransferServer.doSaslHandshake requires the connection to begin with SASL_TRANSFER_MAGIC_NUMBER (0xDEADBEEF, DataTransferSaslUtil.java:86). Any other first int throws InvalidMagicNumberException, formatted as 'Received %x instead of %x from client.'; the exception also records whether the DN runs dfs.encrypt.data-transfer (isHandshake4Encryption).

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/SaslDataTransferServer.java:388

   *
   * @param peer connection peer
   * @param underlyingOut connection output stream
   * @param underlyingIn connection input stream
   * @param saslProps properties of SASL negotiation
   * @param callbackHandler for responding to SASL callbacks
   * @return new pair of streams, wrapped after SASL negotiation
   * @throws IOException for any error
   */
  private IOStreamPair doSaslHandshake(Peer peer, OutputStream underlyingOut,
      InputStream underlyingIn, Map<String, String> saslProps,
      CallbackHandler callbackHandler) throws IOException {

    DataInputStream in = new DataInputStream(underlyingIn);
    DataOutputStream out = new DataOutputStream(underlyingOut);

    int magicNumber = in.readInt();
    if (magicNumber != SASL_TRANSFER_MAGIC_NUMBER) {
      throw new InvalidMagicNumberException(magicNumber, 
          dnConf.getEncryptDataTransfer());
    }
    try {
      // step 1
      SaslMessageWithHandshake message = readSaslMessageWithHandshakeSecret(in);
      byte[] secret = message.getSecret();
      String bpid = message.getBpid();
      Map<String, String> dynamicSaslProps = new TreeMap<>(saslProps);
      if (secret != null || bpid != null) {
        // sanity check, if one is null, the other must also not be null
        assert(secret != null && bpid != null);
        String qop = new String(secret, StandardCharsets.UTF_8);
        saslProps.put(Sasl.QOP, qop);
        dynamicSaslProps.put(Sasl.QOP, qop);
      }
      SaslParticipant sasl = SaslParticipant.createServerSaslParticipant(
          dynamicSaslProps, callbackHandler);

View on GitHub (pinned to 2add963021)

Solutions

1. Configure dfs.data.transfer.protection identically on clients and DataNodes, then restart the clients
2. Upgrade clients that predate SASL data-transfer support
3. Check DataNode logs: these connections are refused by design - identify and remove the non-SASL client dialing the xfer port

Example fix

// before: DN requires SASL, client sends plain protocol bytes -> InvalidMagicNumberException
// after: client hdfs-site.xml matches the DataNodes
<property><name>dfs.data.transfer.protection</name><value>authentication</value></property>

// Client-side preflight before enabling secure data transfer on a DN:
// ensure every DataNode you will talk to also has SASL configured,
// e.g. via dfsadmin/JMX:
//   for dn in $(hdfs dfsadmin -report -live | awk '/Hostname:/{print $2}'); do
//     curl -s "$dn:9864/jmx?qry=Hadoop:service=DataNode,name=DataNodeInfo" \
//       | grep -q dfs.data.transfer.protection || echo "DN missing protection: $dn"
//   done

// Mirror what SaslDataTransferClient does: treat the magic-number failure as a signal
try {
  saslClientNegotiation(dnPeer);
} catch (InvalidMagicNumberException e) {
  if (e.isHandshake4Encryption()) {
    // DN did not even start SASL: it lacks dfs.data.transfer.protection;
    // surface config guidance instead of retrying the connection
  } else {
    // trust chain rejected: report the DN address and its protection settings
  }
}

When it happens

Trigger: A plain (non-SASL) data-transfer client connects to a DataNode that requires SASL negotiation: it sends op/version bytes instead of the magic number, so the first int never equals 0xDEADBEEF. Also garbage/probes on the xfer port.

Common situations: Enabling dfs.data.transfer.protection on DataNodes while some clients (older versions, misconfigured gateways, native tools) do not negotiate SASL; port scanners hitting dfs.datanode.data.port; half-applied security configs.

Related errors

• Cannot create a secured connection if DataNode listens on un
• Provided name '{}' has {} components instead of the expected
• Version Mismatch (Expected: {}, Received: {} )
• Unknown op {} in data stream
• Replica was found but missing fields.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/385a41537edd3fc8. Report an issue: GitHub.