{"record":{"id":"fa10910e43d000f8","repo":"t8y2/dbx","slug":"continue-zookeeper-gssapi-negotiation-at-round-d","errorCode":null,"errorMessage":"continue ZooKeeper GSSAPI negotiation at round %d: %w","messagePattern":"continue ZooKeeper GSSAPI negotiation at round (.+?): %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"agents/drivers/argo-go/zookeeper_protocol.go","lineNumber":226,"sourceCode":"\t\t\tencoder.bytes(token)\n\t\t})\n\t\tif requestErr != nil {\n\t\t\treturn fmt.Errorf(\"ZooKeeper SASL round %d: %w\", round+1, requestErr)\n\t\t}\n\t\tdecoder := newZooKeeperDecoder(response)\n\t\tchallenge, decodeErr := decoder.bytes()\n\t\tif decodeErr != nil {\n\t\t\treturn fmt.Errorf(\"decode ZooKeeper SASL round %d: %w\", round+1, decodeErr)\n\t\t}\n\t\tif saslClient.Complete() {\n\t\t\tif len(challenge) != 0 {\n\t\t\t\treturn errors.New(\"ZooKeeper sent an unexpected token after GSSAPI completion\")\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\t\ttoken, err = saslClient.Step(challenge)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"continue ZooKeeper GSSAPI negotiation at round %d: %w\", round+1, err)\n\t\t}\n\t}\n\treturn fmt.Errorf(\"ZooKeeper GSSAPI negotiation exceeded %d rounds\", zooKeeperMaxSASLRounds)\n}\n\nfunc (client *protocolZooKeeperClient) AddAuth(scheme string, auth []byte) error {\n\t_, err := client.request(zooKeeperOpSetAuth, func(encoder *zooKeeperEncoder) {\n\t\tencoder.int32(0)\n\t\tencoder.string(scheme)\n\t\tencoder.bytes(auth)\n\t})\n\treturn err\n}\n\nfunc (client *protocolZooKeeperClient) Children(path string) ([]string, *zk.Stat, error) {\n\tresponse, err := client.request(zooKeeperOpGetChildren2, func(encoder *zooKeeperEncoder) {\n\t\tencoder.string(path)\n\t\tencoder.boolean(false)","sourceCodeStart":208,"sourceCodeEnd":244,"githubUrl":"https://github.com/t8y2/dbx/blob/c0390bff16418b651f4728520d99adf8ce48829a/agents/drivers/argo-go/zookeeper_protocol.go#L208-L244","documentation":"This error wraps a failure of the local GSSAPI implementation when computing the next SASL token from the server's challenge (saslClient.Step). authenticateSASL got a valid challenge back but the Kerberos/GSSAPI mechanism itself rejected it — e.g. the token is not valid for the current security context, the realm/principal does not match, or replay/credential problems. The client-side mechanism, not the network, failed.","triggerScenarios":"Calling authenticateSASL where saslClient.Step(challenge) returns an error at round N: server returns a GSSAPI error token (e.g. Kerberos AP_ERR replay, wrong principal in service ticket, clock skew beyond ticket skew limits), or the client's credentials expired mid-handshake.","commonSituations":"Clock skew between client and KDC/server exceeding the 5-minute Kerberos tolerance, the service ticket is for the wrong ZooKeeper principal (spnego/zookeeper service name mismatch in krb5.conf), credentials cache expiring during long-running agents, or cross-realm trust misconfiguration.","solutions":["Check clock synchronization (ntp/chrony) between the client, ZooKeeper servers, and the KDC — skew over the Kerberos tolerance causes GSSAPI token rejection.","Verify the service principal the client requests matches the server's registered principal (service name and realm in krb5.conf / the connection string host reverse-resolves to the right realm).","Re-run kinit to refresh expired credentials, and ensure the credential cache path is correct for the process (CCACHE env var, keytab-based refresh for daemons).","Inspect the wrapped inner error from Step — GSSAPI major/minor status codes identify whether it is a replay, principal mismatch, or credential expiry.","If cross-realm, verify the trust path (capaths) or use a principal in the server's own realm."],"exampleFix":"// before: principal mismatch\nprincipal := \"zkclient@WRONG.REALM\" // continue ZooKeeper GSSAPI negotiation at round 1: ... principal unknown\n// after: use the principal registered in the server's JAAS/keytab\nprincipal := \"zookeeper/zk-1.internal@CORRECT.REALM\"","handlingStrategy":"validation","validationCode":"// Check Kerberos prerequisites before attempting SASL\nif _, err := os.Stat(os.Getenv(\"KRB5CCNAME\")); err != nil {\n    if err := exec.Command(\"kinit\", \"-kt\", keytabPath, principal).Run(); err != nil {\n        return fmt.Errorf(\"cannot acquire Kerberos credentials for %s: %w\", principal, err)\n    }\n}\n// verify clock skew\nif skew := clockSkewVsKDC(); skew > 4*time.Minute {\n    return fmt.Errorf(\"clock skew %v vs KDC exceeds Kerberos tolerance\", skew)\n}","typeGuard":null,"tryCatchPattern":"err := client.Connect()\nif err != nil && strings.Contains(err.Error(), \"GSSAPI negotiation\") {\n    // local mechanism rejected the challenge — retrying without re-kinit won't help\n    if refreshErr := kinitFromKeytab(); refreshErr != nil {\n        return refreshErr\n    }\n    return client.Connect()\n}","preventionTips":["Keep client and server clocks synced with NTP/chrony (Kerberos fails on >5min skew).","Verify the service principal in krb5.conf matches the server's registered ZooKeeper principal.","Use keytab-based credential refresh for long-running daemons instead of interactive tickets.","Test GSSAPI independently (kvno <service-principal>) before wiring up the client."],"tags":["go","zookeeper","sasl","kerberos","gssapi","authentication"],"backgroundTag":"kerberos-authentication-failed","analyzedSha":"c0390bff16418b651f4728520d99adf8ce48829a","analyzedAt":"2026-09-05T23:05:10.900Z","contentChangedAt":"2026-09-05T23:05:10.900Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}