{"record":{"id":"d6c21e3b9649f5fb","repo":"t8y2/dbx","slug":"no-available-hiveserver2-nodes-in-zookeeper-namesp","errorCode":null,"errorMessage":"no available HiveServer2 nodes in ZooKeeper namespace %s","messagePattern":"no available HiveServer2 nodes in ZooKeeper namespace (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"agents/drivers/argo-go/discovery.go","lineNumber":160,"sourceCode":"\t\t\tif parseErr == nil {\n\t\t\t\tresolved = append(resolved, value)\n\t\t\t} else {\n\t\t\t\tnodeFailures = append(nodeFailures, fmt.Sprintf(\"%s/%s: %v\", path, child, parseErr))\n\t\t\t}\n\t\t}\n\t\tif len(resolved) > 0 {\n\t\t\tbreak\n\t\t}\n\t}\n\tresolved = shuffledEndpoints(uniqueEndpoints(resolved), rejected)\n\tif len(resolved) == 0 {\n\t\tif listedPath == \"\" {\n\t\t\treturn nil, fmt.Errorf(\"HiveServer2 ZooKeeper namespace not found; tried %s\", strings.Join(discovery.paths(), \", \"))\n\t\t}\n\t\tif len(nodeFailures) > 0 {\n\t\t\treturn nil, fmt.Errorf(\"no usable HiveServer2 nodes in ZooKeeper namespace %s: %s\", listedPath, strings.Join(nodeFailures, \"; \"))\n\t\t}\n\t\treturn nil, fmt.Errorf(\"no available HiveServer2 nodes in ZooKeeper namespace %s\", listedPath)\n\t}\n\treturn resolved, nil\n}\n\nfunc (discovery *zooKeeperDiscovery) paths() []string {\n\tnamespace := strings.Trim(discovery.namespace, \"/\")\n\tif strings.EqualFold(discovery.discoveryMode, \"zookeeperha\") {\n\t\treturn []string{\n\t\t\tzooKeeperPath(namespace, \"instances\"),\n\t\t\tzooKeeperPath(namespace+\"-unsecure\", \"instances\"),\n\t\t\tzooKeeperPath(namespace+\"-sasl\", \"instances\"),\n\t\t}\n\t}\n\treturn []string{zooKeeperPath(namespace)}\n}\n\nfunc zooKeeperPath(parts ...string) string {\n\tcleaned := make([]string, 0, len(parts))","sourceCodeStart":142,"sourceCodeEnd":178,"githubUrl":"https://github.com/t8y2/dbx/blob/c0390bff16418b651f4728520d99adf8ce48829a/agents/drivers/argo-go/discovery.go#L142-L178","documentation":"Endpoints() found the ZooKeeper namespace (Children listed successfully) but every child resolved to zero unique endpoints without recording any failure — typically the namespace simply contains no children, or all children were filtered out as already-rejected duplicates. Distinct from the 'not found' case: the path exists but is empty (or everything was rejected).","triggerScenarios":"Children(path) returned an empty list, or all resolved endpoints were filtered by uniqueEndpoints/shuffledEndpoints(rejected) because every known server address was previously marked rejected.","commonSituations":"All HiveServer2 instances are down or just deregistered from ZK; discovery retry loop has marked every endpoint as rejected after failed connection attempts; namespace was created but HiveServer2 has not registered yet (startup race).","solutions":["Confirm HiveServer2 instances are running and registered: zkCli ls /<namespace> should show children.","If this is a retry after connection failures, check the HiveServer2 hosts directly — they were all previously rejected as unreachable.","Wait/retry briefly if HiveServer2 is starting up; registration happens after server readiness.","Verify the same ZK ensemble is used by both HiveServer2 and the client config."],"exampleFix":"// before: hammering a rejected set forever\nendpoints, err := discovery.Endpoints(ctx, rejected) // all endpoints rejected\n// after: clear rejected set after a cool-down\nif len(rejected) > 0 && time.Since(lastReset) > 30*time.Second {\n    rejected = map[string]bool{}\n}","handlingStrategy":"fallback","validationCode":"// preflight: namespace should contain children\nconn, _, _ := zk.Connect(zkHosts, 5*time.Second)\nchildren, _, err := conn.Children(\"/\"+ns)\nconn.Close()\nif err != nil || len(children) == 0 {\n    return fmt.Errorf(\"namespace /%s has no registered HiveServer2 instances\", ns)\n}","typeGuard":null,"tryCatchPattern":"eps, err := discovery.Endpoints(ctx, rejected)\nif err != nil && strings.Contains(err.Error(), \"no available HiveServer2 nodes\") {\n    // fall back to a static endpoint list while ZK has nothing registered\n    return staticEndpoints, nil\n}","preventionTips":["Keep at least one HiveServer2 instance running and registered.","Clear or periodically reset the rejected-endpoint set in retry loops so healthy servers are not permanently excluded.","Add a small startup delay/retry for HiveServer2 registration lag.","Alert when the discovery znode child count drops to zero."],"tags":["zookeeper","service-discovery","availability","go"],"backgroundTag":"zookeeper-namespace-empty","analyzedSha":"c0390bff16418b651f4728520d99adf8ce48829a","analyzedAt":"2026-09-05T23:05:10.900Z","contentChangedAt":"2026-09-05T23:05:10.900Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}