{"record":{"id":"8cd2c75442f54644","repo":"t8y2/dbx","slug":"zookeeper-sent-an-unexpected-token-after-digest-md","errorCode":null,"errorMessage":"ZooKeeper sent an unexpected token after DIGEST-MD5 completion","messagePattern":"ZooKeeper sent an unexpected token after DIGEST-MD5 completion","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"agents/drivers/zookeeper/sasl.go","lineNumber":101,"sourceCode":"\t\ttimeout = defaultConnectionTimeout\n\t}\n\tif err := connection.SetDeadline(time.Now().Add(timeout)); err != nil {\n\t\treturn err\n\t}\n\tdefer connection.SetDeadline(time.Time{})\n\n\ttoken, err := saslClient.Start()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"start ZooKeeper DIGEST-MD5 negotiation: %w\", err)\n\t}\n\tfor round := 0; round < zooKeeperSASLMaxRounds; round++ {\n\t\tchallenge, err := zooKeeperSASLRound(connection, zooKeeperSASLXIDBase+int32(round), token)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"ZooKeeper SASL round %d: %w\", round+1, err)\n\t\t}\n\t\tif saslClient.Complete() {\n\t\t\tif len(challenge) != 0 {\n\t\t\t\treturn errors.New(\"ZooKeeper sent an unexpected token after DIGEST-MD5 completion\")\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\t\ttoken, err = saslClient.Step(challenge)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"continue ZooKeeper DIGEST-MD5 negotiation at round %d: %w\", round+1, err)\n\t\t}\n\t\tif saslClient.Complete() {\n\t\t\tif len(token) != 0 {\n\t\t\t\treturn errors.New(\"ZooKeeper DIGEST-MD5 completed with an unexpected client token\")\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\t}\n\treturn fmt.Errorf(\"ZooKeeper DIGEST-MD5 negotiation exceeded %d rounds\", zooKeeperSASLMaxRounds)\n}\n\nfunc zooKeeperSASLRound(connection net.Conn, xid int32, token []byte) ([]byte, error) {","sourceCodeStart":83,"sourceCodeEnd":119,"githubUrl":"https://github.com/t8y2/dbx/blob/c0390bff16418b651f4728520d99adf8ce48829a/agents/drivers/zookeeper/sasl.go#L83-L119","documentation":"negotiateSASLDigest runs the DIGEST-MD5 challenge rounds. Once the SASL client reports Complete(), the protocol expects the server to stop issuing challenges; if another non-empty token arrives after completion, the driver aborts with 'ZooKeeper sent an unexpected token after DIGEST-MD5 completion' because the handshake deviates from RFC 2831/the ZooKeeper SASL exchange.","triggerScenarios":"During authenticateSASLDigest, after saslClient.Complete() is true, zooKeeperSASLRound still returns a non-empty challenge — the server sent an extra token after the final client response. Raised in agents/drivers/zookeeper/sasl.go:101.","commonSituations":"Server-side SASL implementation or version behaving non-strictly (extra post-completion token); a proxy/middleware in front of ZooKeeper injecting an extra frame; server misconfigured with layered security (qop) producing an additional challenge; version mismatch between client library expectations and server SASL behavior.","solutions":["Check the ZooKeeper server version and SASL configuration (jaas.conf, digest-md5 settings) for known quirks; upgrade to a version with correct DIGEST-MD5 termination.","Remove/inspect any proxy between client and ensemble that could inject extra frames.","Verify the negotiated protection quality (qop) matches on both sides — enable/auth/integrity mismatches can produce extra tokens.","Capture the handshake (tcpdump/Wireshark on port 2181) and compare token counts against a known-good zkCli SASL session."],"exampleFix":"// server jaas.conf before\n// ZooKeeper { org.apache.zookeeper.server.auth.DigestLoginModule required ... user_super=\"pw\"; } // missing/broken entry causes odd SASL exchange\n// after\nZooKeeper { org.apache.zookeeper.server.auth.DigestLoginModule required user_svc=\"correct-pw\"; };","handlingStrategy":"try-catch","validationCode":"// Pre-flight SASL sanity: confirm server advertises digest-md5 and credentials exist\n// zkCli.sh with matching jaas.conf should authenticate without errors before app start","typeGuard":null,"tryCatchPattern":"err := authenticateSASLDigest(conn, creds)\nif err != nil {\n    if strings.Contains(err.Error(), \"unexpected token after DIGEST-MD5 completion\") {\n        // server/proxy misbehaving; log details and fall back or fail with context\n        return fmt.Errorf(\"server sent extra SASL token; check ZooKeeper version/jaas.conf and any proxy: %w\", err)\n    }\n    return err\n}","preventionTips":["Pin a ZooKeeper server version known to terminate DIGEST-MD5 cleanly","Avoid proxies/middlewares on the SASL path or test them for frame injection","Align qop (protection quality) settings on client and server","Validate SASL login with zkCli against the same ensemble before rollout"],"tags":["sasl","zookeeper","authentication","protocol"],"backgroundTag":"sasl-handshake-failed","analyzedSha":"c0390bff16418b651f4728520d99adf8ce48829a","analyzedAt":"2026-09-05T23:05:10.900Z","contentChangedAt":"2026-09-05T23:05:10.900Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}