thanos-io/thanos · error

invalid label name

Error message

invalid label name: %q

What it means

createAttributes translates OTLP attribute keys into Prometheus label names via labelNamer.Build. When a metric attribute name cannot be converted to a valid Prometheus label name (after all normalization attempts fail), this error wraps the underlying naming failure and aborts translation of the whole metric.

Solutions

1. Rename the OTLP attribute key in the emitting application to match Prometheus label name rules ([a-zA-Z_][a-zA-Z0-9_]*)
2. Sanitize keys at the collector/SDK side before export
3. Enable/verify the label-naming normalization settings (e.g. AllowUTF8 off means strict legacy names) and ensure keys pass it
4. Log and drop the offending attribute server-side instead of failing the metric

Example fix

// before: attribute with key "2xx-total"
attrs.Add("2xx-total", n)
// after: valid label name
attrs.Add("_2xx_total", n)

// Go: validate attribute keys against label-name rules before export
var labelRe = regexp.MustCompile(`^[a-zA-Z_][a-zA-Z0-9_]*$`)
func validLabelName(k string) bool { return labelRe.MatchString(k) }

func isSafeLabelKey(k string) bool {
    return k != "" && labelRe.MatchString(k)
}

attrs, err := createAttributes(...)
if err != nil {
    if strings.Contains(err.Error(), "invalid label name") {
        log.Warnf("dropping metric with bad attribute: %v", err)
        return nil // or sanitize and retry
    }
    return err
}

When it happens

Trigger: Sending OTLP metrics whose attribute key, even after sanitization by the namer, cannot become a valid label name — e.g. an attribute key starting with a digit after normalization, or an empty/entirely invalid key, in AddGauge/Sum/Histogram/Summary/ExponentialHistogram data point translation.

Common situations: Instrumented apps using numeric keys like {'2xx_requests': ...} or keys composed solely of special characters; proxies/SDKs forwarding raw telemetry with non-ASCII keys; tenant sending attributes violating Prometheus naming rules with TranslateOTLP without allow-utf8.

Understand the failure class

Background: "invalid id" errors: invalid identifier format — why libraries reject IDs before lookup, and how to fix them — this error's family across 37 libraries.

Related errors

• invalid extra label name
• unsupported format for label
• invalid label name
• split queries interval should be greater than 0 when…
• labels.default-time-range cannot be set to 0

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/b44b18a2dd45640c. Report an issue: GitHub.

Appendix: source

Thrown at pkg/receive/otlptranslator/helper.go:157

	// XXX: Should we always drop service namespace/service name/service instance ID from the labels
	// (as they get mapped to other Prometheus labels)?
	attributes.Range(func(key string, value pcommon.Value) bool {
		if !slices.Contains(ignoreAttrs, key) {
			labels = append(labels, labelpb.ZLabel{Name: key, Value: value.AsString()})
		}
		return true
	})
	sort.Stable(ByLabelName(labels))

	labelNamer := prometheustranslator.LabelNamer{
		UTF8Allowed: settings.AllowUTF8,
	}
	// map ensures no duplicate label names.
	l := make(map[string]string, maxLabelCount)
	for _, label := range labels {
		finalKey, err := labelNamer.Build(label.Name)
		if err != nil {
			return nil, errors.Wrapf(err, "invalid label name: %q", label.Name)
		}
		if existingValue, alreadyExists := l[finalKey]; alreadyExists {
			l[finalKey] = existingValue + ";" + label.Value
		} else {
			l[finalKey] = label.Value
		}
	}

	for _, lbl := range promotedAttrs {
		normalized, err := labelNamer.Build(lbl.Name)
		if err != nil {
			return nil, errors.Wrapf(err, "invalid promoted resource attribute name: %q", lbl.Name)
		}
		if _, exists := l[normalized]; !exists {
			l[normalized] = lbl.Value
		}
	}

View on GitHub (pinned to 35b8b99117)