thanos-io/thanos · error

parsing metric families against

Error message

parsing metric families against %s

What it means

After fetching /metrics, LowestTimestamp parses the exposition text with expfmt.TextToMetricFamilies. If the body is not valid Prometheus text exposition format (truncated response, HTML error page, proxy interference, malformed metrics), the parse error is wrapped with "parsing metric families against <url>".

Solutions

1. curl the /metrics URL and confirm the body is Prometheus text exposition format, not HTML.
2. Check any reverse proxy/auth layer in front of Prometheus and exempt /metrics from HTML interstitials.
3. Verify no compression/content-encoding mismatch; ensure the client is not handing a gzip stream to the text parser un-decoded.
4. Confirm the port actually serves Prometheus, not another service.

Example fix

// debugging aid: inspect what was actually returned
resp, err := c.Do(req.WithContext(ctx))
if err != nil { ... }
body, _ := io.ReadAll(resp.Body)
if !bytes.HasPrefix(bytes.TrimSpace(body), []byte("#")) {
	return fmt.Errorf("/metrics at %s did not return exposition text: %.200s", u.String(), body)
}
families, err := parser.TextToMetricFamilies(bytes.NewReader(body))

resp, err := http.Get(metricsURL)
if err != nil { return err }
body, _ := io.ReadAll(io.LimitReader(resp.Body, 512))
if !bytes.Contains(body, []byte("# HELP")) && !bytes.Contains(body, []byte("# TYPE")) {
	return fmt.Errorf("%s does not serve Prometheus exposition text", metricsURL)
}

func servesExpositionFormat(body []byte) bool {
	return bytes.HasPrefix(bytes.TrimSpace(body), []byte("#"))
}

When it happens

Trigger: parser.TextToMetricFamilies(resp.Body) fails because the response body is not valid Prometheus text format: an HTML login/error page from a reverse proxy, truncated body, gzip/compression mismatch, or invalid metric syntax.

Common situations: A proxy or auth gateway returns an HTML 200 page instead of metrics; Prometheus exposes protobuf format unexpectedly; middleboxes truncate large /metrics responses; non-Prometheus service listening on the configured port.

Understand the failure class

Background: "Invalid JSON response" and "Failed to parse response" errors: when an API answers 200 but the body isn't the JSON your library expected — this error's family across 28 libraries.

Related errors

• failed to validate prometheus flags
• failed to get prometheus version
• initial external labels query
• empty name for metric family
• non-unique name for metric family

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/8b4bce2cb31d1b2b. Report an issue: GitHub.

Appendix: source

Thrown at pkg/promclient/promclient.go:722

	req, err := http.NewRequest(http.MethodGet, u.String(), nil)
	if err != nil {
		return 0, errors.Wrap(err, "create request")
	}

	span, ctx := tracing.StartSpan(ctx, "/lowest_timestamp HTTP[client]")
	defer span.Finish()

	resp, err := c.Do(req.WithContext(ctx))
	if err != nil {
		return 0, errors.Wrapf(err, "request metric against %s", u.String())
	}
	defer runutil.ExhaustCloseWithLogOnErr(c.logger, resp.Body, "request body")

	parser := expfmt.NewTextParser(model.UTF8Validation)
	families, err := parser.TextToMetricFamilies(resp.Body)
	if err != nil {
		return 0, errors.Wrapf(err, "parsing metric families against %s", u.String())
	}
	mf, ok := families["prometheus_tsdb_lowest_timestamp_seconds"]
	if !ok {
		return 0, errors.Wrapf(err, "metric families did not contain 'prometheus_tsdb_lowest_timestamp_seconds'")
	}
	val := 1000 * mf.GetMetric()[0].GetGauge().GetValue()

	// in the case that we dont have cut a block yet, TSDB lowest timestamp is math.MaxInt64
	// but its represented as float and truncated so we need to do this weird comparison.
	// Since we use this for fan-out pruning we use min timestamp here to include this prometheus.
	if val == float64(math.MaxInt64) {
		return math.MinInt64, nil
	}
	return int64(val), nil
}

func formatTime(t time.Time) string {
	return strconv.FormatFloat(float64(t.Unix())+float64(t.Nanosecond())/1e9, 'f', -1, 64)

View on GitHub (pinned to 35b8b99117)