vitessio/vitess · error

not ok: %v

Error message

not ok: %v

What it means

This is the HTTP 500 body returned by the vttablet /debug/health endpoint when tsv.IsHealthy() returns an error. Unlike /healthz, this check actually verifies the tablet can run a query (usually a ping against MySQL), so the error message wraps the underlying health failure (e.g. connection to MySQL failed, query timeout, tablet not serving). It indicates the tablet cannot currently process queries end-to-end.

Source

Thrown at go/vt/vttablet/tabletserver/tabletserver.go:2101

		http.Error(w, "500 internal server error: vttablet is not serving", http.StatusInternalServerError)
		return
	}
	w.Header().Set("Content-Length", strconv.Itoa(len(okMessage)))
	w.WriteHeader(http.StatusOK)
	w.Write(okMessage)
}

// Query service health check
// Returns ok if a query can go all the way to database and back
func (tsv *TabletServer) registerDebugHealthHandler() {
	tsv.exporter.HandleFunc("/debug/health", func(w http.ResponseWriter, r *http.Request) {
		if err := acl.CheckAccessHTTP(r, acl.MONITORING); err != nil {
			acl.SendError(w, err)
			return
		}
		w.Header().Set("Content-Type", "text/plain")
		if err := tsv.IsHealthy(); err != nil {
			http.Error(w, fmt.Sprintf("not ok: %v", err), http.StatusInternalServerError)
			return
		}
		w.Write([]byte("ok"))
	})
}

func (tsv *TabletServer) registerQueryzHandler() {
	tsv.exporter.HandleFunc("/queryz", func(w http.ResponseWriter, r *http.Request) {
		queryzHandler(tsv.qe, w, r)
	})
}

func (tsv *TabletServer) registerQuerylogzHandler() {
	tsv.exporter.HandleFunc("/querylogz", func(w http.ResponseWriter, r *http.Request) {
		ch := tabletenv.StatsLogger.Subscribe("querylogz")
		defer tabletenv.StatsLogger.Unsubscribe(ch)
		querylogzHandler(ch, w, r, tsv.env.Parser())
	})

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Read the error detail after 'not ok:' in the response — it names the underlying health failure — and fix that cause (restore MySQL connectivity, address replication lag/timeout).
2. Check vttablet logs around the failing health check for the full IsHealthy error and any MySQL errors.
3. Confirm mysqld is up (mysqlctl status) and that the tablet can reconnect; restart vttablet if it is stuck in a failed state.
4. If this happens under load only, increase the health check tolerance or scale MySQL resources; if it persists, take the tablet out of rotation and investigate.

Example fix

// before: ignoring the detailed error in monitoring
if !strings.HasSuffix(body, "ok") { alert("tablet down") }
// after: parse and surface the underlying cause
if !strings.HasSuffix(body, "ok") {
    cause := strings.TrimPrefix(body, "not ok: ")
    alert("tablet down: " + cause) // e.g. 'connection to mysql failed'
}

// Probe deep health before sending queries
type HealthChecker interface { IsHealthy() error }
if err := h.IsHealthy(); err != nil {
    return fmt.Errorf("tablet deep health failed: %w", err)
}

func isDeepHealthError(body string) bool {
    return strings.HasPrefix(body, "not ok: ")
}
// if isDeepHealthError(resp.Body) { parse the cause after 'not ok: ' }

body, err := io.ReadAll(resp.Body)
if resp.StatusCode != http.StatusOK && strings.HasPrefix(string(body), "not ok: ") {
    cause := strings.TrimPrefix(string(body), "not ok: ")
    return fmt.Errorf("deep health check failed: %s", cause)
}

When it happens

Trigger: An HTTP GET to /debug/health (with acl MONITORING access) while tsv.IsHealthy() fails — typically because the MySQL backend is unreachable, the health check query times out, the tabletserver is not serving, or transactions/queries are in a bad state (e.g. during shutdown).

Common situations: mysqld down or restarted; network partition between vttablet and MySQL; MySQL overloaded so the health query exceeds the timeout; monitoring/consul health checks flipping the tablet out of the serving set; acl misconfiguration blocking the request (returns a different acl error though).

Related errors

• tablet %s is no longer healthy: %s, restarting vstream
• tablet is not healthy. tablet: %v health record: %v
• 500 internal server error: vttablet is not serving
• no client certs for connection
• unexpected: query ended without no results and no error

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/0f3dbcc64b225ed4. Report an issue: GitHub.