{"record":{"id":"869a3f89634dacf1","repo":"t8y2/dbx","slug":"all-hiveserver2-endpoints-failed-s-869a3f","errorCode":null,"errorMessage":"all HiveServer2 endpoints failed: %s","messagePattern":"all HiveServer2 endpoints failed: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"agents/drivers/hive-go/connector.go","lineNumber":167,"sourceCode":"\t\t\t\trejected[target.address()] = true\n\t\t\t\tfailures = append(failures, fmt.Sprintf(\"attempt %d %s: %v\", attempt+1, target.address(), connectErr))\n\t\t\t}\n\t\t\tbreak\n\t\t}\n\t\tif attempt+1 < connector.retries && connector.retryInterval > 0 {\n\t\t\ttimer := time.NewTimer(connector.retryInterval)\n\t\t\tselect {\n\t\t\tcase <-ctx.Done():\n\t\t\t\ttimer.Stop()\n\t\t\t\treturn nil, ctx.Err()\n\t\t\tcase <-timer.C:\n\t\t\t}\n\t\t}\n\t}\n\tif len(failures) == 0 {\n\t\treturn nil, errors.New(\"Hive discovery returned no endpoints\")\n\t}\n\treturn nil, fmt.Errorf(\"all HiveServer2 endpoints failed: %s\", strings.Join(failures, \"; \"))\n}\n\nfunc (connector *discoveryConnector) Driver() driver.Driver {\n\treturn connector.driver\n}\n\nfunc openHiveDatabase(config connectionConfig) *sql.DB {\n\tdatabase := sql.OpenDB(newDiscoveryConnector(config))\n\tdatabase.SetMaxOpenConns(1)\n\tdatabase.SetMaxIdleConns(1)\n\treturn database\n}\n\nfunc normalizeHiveAuth(value string) string {\n\tnormalized := strings.ToUpper(strings.TrimSpace(value))\n\tswitch normalized {\n\tcase \"\":\n\t\treturn \"NONE\"","sourceCodeStart":149,"sourceCodeEnd":185,"githubUrl":"https://github.com/t8y2/dbx/blob/c0390bff16418b651f4728520d99adf8ce48829a/agents/drivers/hive-go/connector.go#L149-L185","documentation":"This error is returned by discoveryConnector.Connect (connector.go:167) after the driver tried every HiveServer2 endpoint returned by ZooKeeper-based service discovery and each connection attempt failed. The message aggregates all per-endpoint failures, joined with \"; \", so the underlying causes (timeouts, refusals, TLS errors) are visible in the message.","triggerScenarios":"Using ZooKeeper discovery mode, discovery returns >=1 endpoints, every endpoint connect attempt fails, and the failures list is non-empty (an empty list yields 'Hive discovery returned no endpoints' instead).","commonSituations":"All HiveServer2 instances down or being restarted; network/firewall blocking the driver from the cluster; stale ZooKeeper node entries pointing at dead hosts; TLS configuration mismatch with all servers.","solutions":["Read the joined failure details to identify the common cause (timeout vs refused vs TLS).","Verify HiveServer2 instances are up and reachable (nc/curl the host:port).","Check network/firewall rules between the client and the Hive cluster.","Confirm ZooKeeper znodes contain current, healthy server addresses."],"exampleFix":"// before (stale znode pointing at dead host)\nzookeeper=zk1:2181/hiveserver2\n// after (verify HS2 is healthy and re-registered)\nhive --service hiveserver2 # restart so it re-registers in ZooKeeper","handlingStrategy":"retry","validationCode":"addrs := strings.Split(host, \":\")\nfor _, a := range zkEndpoints {\n    conn, err := net.DialTimeout(\"tcp\", a, 3*time.Second)\n    if err != nil {\n        return fmt.Errorf(\"zk endpoint unreachable: %s\", a)\n    }\n    conn.Close()\n}\n","typeGuard":null,"tryCatchPattern":"conn, err := db.Connect(ctx)\nif err != nil && strings.Contains(err.Error(), \"all HiveServer2 endpoints failed\") {\n    // inspect per-endpoint causes in err.Error(), then retry with backoff\n    return retryWithBackoff(ctx, 3, db.Connect)\n}\n","preventionTips":["Monitor HiveServer2 health and keep at least one healthy instance registered in ZooKeeper.","TTL-clean stale znode entries so discovery does not return dead hosts.","Set explicit connect timeouts so failover fails fast with clear per-endpoint causes.","Alert on the aggregated failure message since it indicates cluster-wide unreachability."],"tags":["connection","zookeeper","failover","go"],"backgroundTag":"all-endpoints-failed","analyzedSha":"c0390bff16418b651f4728520d99adf8ce48829a","analyzedAt":"2026-09-05T23:05:10.900Z","contentChangedAt":"2026-09-05T23:05:10.900Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}