thanos-io/thanos · error

unmarshal response

Error message

unmarshal response: %v

What it means

ExternalLabels fetches /api/v1/status/config from Prometheus and unmarshals the JSON response into a struct with a data.yaml field. This error wraps json.Unmarshal failures, including the offending body in the message, so a malformed or unexpected JSON payload can be diagnosed.

Solutions

1. Check the body printed in the error to see what was actually returned
2. Verify the URL points at a real Prometheus /api/v1/status/config endpoint (curl it)
3. Remove/fix proxies returning HTML instead of JSON
4. Check Prometheus version compatibility for the status/config endpoint

Example fix

// before: hard to know what came back
curl -s http://prometheus:9090/api/v1/status/config | head
// after: confirm valid JSON shape
{"status":"success","data":{"yaml":"global: ..."}}

labels, err := client.ExternalLabels(ctx, baseURL)
if err != nil {
    if strings.Contains(err.Error(), "unmarshal response") {
        // body wasn't JSON: check for proxies/auth pages and log the body
        log.Printf("non-JSON response from status/config: %v", err)
    }
    return err
}

When it happens

Trigger: The body returned by the Prometheus /api/v1/status/config endpoint is not valid JSON or has a different shape than {data:{yaml:string}} — e.g. an HTML error page from a proxy, an auth redirect, or a truncated response. Called from UpdateLabels.

Common situations: A reverse proxy in front of Prometheus returns an HTML login page (200 with HTML); an older/other API returns a different JSON schema; gzip/mangling of the body; hitting the wrong port where another service listens.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

• unmarshal query range response
• unmarshal query instant response
• decode result into ValueTypeMatrix
• unmarshal alertmanager alert API response
• unmarshal build info API response

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/198b67a3ec26a242. Report an issue: GitHub.

Appendix: source

Thrown at pkg/promclient/promclient.go:201

// Note that configuration can be hot reloadable on Prometheus, so this config might change in runtime.
func (c *Client) ExternalLabels(ctx context.Context, base *url.URL) (labels.Labels, error) {
	u := *base
	u.Path = path.Join(u.Path, "/api/v1/status/config")

	span, ctx := tracing.StartSpan(ctx, "/prom_config HTTP[client]")
	defer span.Finish()

	body, _, err := c.req2xx(ctx, &u, http.MethodGet, nil)
	if err != nil {
		return labels.EmptyLabels(), err
	}
	var d struct {
		Data struct {
			YAML string `json:"yaml"`
		} `json:"data"`
	}
	if err := json.Unmarshal(body, &d); err != nil {
		return labels.EmptyLabels(), errors.Wrapf(err, "unmarshal response: %v", string(body))
	}
	var cfg struct {
		GlobalConfig struct {
			ExternalLabels map[string]string `yaml:"external_labels"`
		} `yaml:"global"`
	}
	if err := yaml.Unmarshal([]byte(d.Data.YAML), &cfg); err != nil {
		return labels.EmptyLabels(), errors.Wrapf(err, "parse Prometheus config: %v", d.Data.YAML)
	}

	return labels.FromMap(cfg.GlobalConfig.ExternalLabels), nil
}

type Flags struct {
	TSDBPath           string         `json:"storage.tsdb.path"`
	TSDBRetention      model.Duration `json:"storage.tsdb.retention"`
	TSDBMinTime        model.Duration `json:"storage.tsdb.min-block-duration"`
	TSDBMaxTime        model.Duration `json:"storage.tsdb.max-block-duration"`

View on GitHub (pinned to 35b8b99117)