{"record":{"id":"60ab86795fb018bf","repo":"apache/hadoop","slug":"failed-to-complete-sasl-handshake","errorCode":null,"errorMessage":"Failed to complete SASL handshake","messagePattern":"Failed to complete SASL handshake","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/DataTransferSaslUtil.java","lineNumber":100,"sourceCode":"  /**\n   * Sent by clients and validated by servers. We use a number that's unlikely\n   * to ever be sent as the value of the DATA_TRANSFER_VERSION.\n   */\n  public static final int SASL_TRANSFER_MAGIC_NUMBER = 0xDEADBEEF;\n\n  /**\n   * Checks that SASL negotiation has completed for the given participant, and\n   * the negotiated quality of protection is included in the given SASL\n   * properties and therefore acceptable.\n   *\n   * @param sasl participant to check\n   * @param saslProps properties of SASL negotiation\n   * @throws IOException for any error\n   */\n  public static void checkSaslComplete(SaslParticipant sasl,\n      Map<String, String> saslProps) throws IOException {\n    if (!sasl.isComplete()) {\n      throw new IOException(\"Failed to complete SASL handshake\");\n    }\n    Set<String> requestedQop = ImmutableSet.copyOf(Arrays.asList(\n        saslProps.get(Sasl.QOP).split(\",\")));\n    String negotiatedQop = sasl.getNegotiatedQop();\n    LOG.debug(\"{}: Verifying QOP: requested = {}, negotiated = {}\",\n        sasl, requestedQop, negotiatedQop);\n    // Treat null negotiated QOP as \"auth\" for the purpose of verification\n    // Code elsewhere does the same implicitly\n    if(negotiatedQop == null) {\n      negotiatedQop = \"auth\";\n    }\n    if (!requestedQop.contains(negotiatedQop)) {\n      throw new IOException(String.format(\"SASL handshake completed, but \" +\n          \"channel does not have acceptable quality of protection, \" +\n          \"requested = %s, negotiated(effective) = %s\", requestedQop, negotiatedQop));\n    }\n  }\n","sourceCodeStart":82,"sourceCodeEnd":118,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/DataTransferSaslUtil.java#L82-L118","documentation":"On a secured cluster (Kerberos or DIGEST-MD5 token authentication with data transfer protection), checkSaslComplete() verifies after SASL negotiation that the SaslParticipant actually completed the handshake. isComplete()==false means the SASL mechanism aborted mid-exchange - credentials were rejected (invalid or expired token/password) or the socket was cut - so the data connection cannot be used and an IOException is thrown.","triggerScenarios":"SaslDataTransferClient/Server negotiate a protected data connection; the peer SASL server returns an error or closes the connection during the challenge/response steps; checkSaslComplete(sasl, saslProps) then finds !sasl.isComplete().","commonSituations":"Expired or non-renewed delegation tokens on long-running jobs; token password mismatch after failover or token re-creation; Kerberos/keytab problems on the DataNode; interrupted sockets during rolling upgrades.","solutions":["Check DataNode logs - the underlying SASL error (rejected credential) is only visible there in detail","Refresh credentials: re-login via keytab or obtain a new delegation token, then retry the operation","Make dfs.data.transfer.protection identical on NameNode, DataNodes and client, and restart/reload configs","Align Hadoop versions between client and DataNodes if the negotiation sequence differs"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  dfsDataOperation();\n} catch (IOException e) {\n  if (e.getMessage() != null\n      && e.getMessage().contains(\"Failed to complete SASL handshake\")) {\n    // credentials rejected mid-negotiation: refresh token / re-login, retry once\n    refreshCredentialsAndRetry();\n  } else {\n    throw e;\n  }\n}","preventionTips":["Renew delegation tokens before expiry on long-running jobs","Keep dfs.data.transfer.protection identical across NN, DNs and clients","Correlate with DataNode logs - the root SASL reason is only logged there"],"tags":["hdfs","sasl","security","data-transfer-protection","authentication"],"backgroundTag":"sasl-handshake-failure","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}