VictoriaMetrics/VictoriaMetrics · error

cannot write tenant %q: %w

Error message

cannot write tenant %q: %w

What it means

Fires in processTenants while streaming the list of tenant names back to vmselect: one of the writeString calls on the RPC connection failed. The tenants query itself succeeded; only the response transport broke, so the caller receives the write error instead of tenant data.

Source

Thrown at lib/vmselectapi/server.go:919

	// Execute the request
	tenants, err := s.api.Tenants(ctx.qt, tr, ctx.deadline)
	if err != nil {
		return ctx.writeErrorMessage(err)
	}

	// Send an empty error message to vmselect.
	if err := ctx.writeString(""); err != nil {
		return fmt.Errorf("cannot send empty error message: %w", err)
	}

	// Send tenants to vmselect
	for _, tenant := range tenants {
		if len(tenant) == 0 {
			logger.Panicf("BUG: unexpected empty tenant name")
		}
		if err := ctx.writeString(tenant); err != nil {
			return fmt.Errorf("cannot write tenant %q: %w", tenant, err)
		}
	}
	// Send 'end of response' marker
	if err := ctx.writeString(""); err != nil {
		return fmt.Errorf("cannot send 'end of response' marker")
	}
	return nil
}

func writeTSDBStatus(ctx *vmselectRequestCtx, status *storage.TSDBStatus) error {
	if err := ctx.writeUint64(status.TotalSeries); err != nil {
		return fmt.Errorf("cannot write totalSeries to vmselect: %w", err)
	}
	if err := ctx.writeUint64(status.TotalLabelValuePairs); err != nil {
		return fmt.Errorf("cannot write totalLabelValuePairs to vmselect: %w", err)
	}
	if err := writeTopHeapEntries(ctx, status.SeriesCountByMetricName); err != nil {
		return fmt.Errorf("cannot write seriesCountByMetricName to vmselect: %w", err)

View on GitHub (pinned to 5079fb58f1)

Solutions

1. Check for network interruptions between vmstorage and vmselect
2. Verify vmselect is still running and has not closed the RPC connection
3. Review logs for preceding timeout or connection-reset errors
4. Retry the tenants request; the error is typically transient

When it happens

Trigger: Thrown at lib/vmselectapi/server.go:919 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of VictoriaMetrics/VictoriaMetrics@5079fb58f1 (2026-09-03). Data as JSON: /api/errors/50c2f6d52657828f. Report an issue: GitHub.