thanos-io/thanos · error

marshal labels

Error message

marshal labels

What it means

After allocating the label list, BuildInto calls marshalLabels to copy each promqltype label (name/value) into the capnp list. Any failure inside that copy (string allocation failure, invalid label data) is wrapped as 'marshal labels'.

Solutions

1. Reduce batch size to shrink per-message allocation pressure
2. Sanitize label names/values (non-empty, sane length) before marshaling
3. Apply metric relabeling to trim label cardinality
4. Check capnp library version for string-set size constraints

Example fix

// before
for _, l := range ts.Labels { /* pass through raw */ }
// after
for _, l := range ts.Labels {
    if l.Name == "" || len(l.Value) > 4096 { continue }
    // marshal
}

func labelsValid(ts prompb.TimeSeries) bool {
    for _, l := range ts.Labels {
        if l.Name == "" || len(l.Name) > 512 || len(l.Value) > 8192 { return false }
    }
    return true
}

if err := BuildInto(tuple, tenant, tsreq, builder); err != nil && strings.Contains(err.Error(), "marshal labels") {
    // sanitize or drop the offending series and continue
    return rebuildWithoutBadSeries(tsreq)
}

When it happens

Trigger: marshalLabels fails while setting capnp text fields for a label — usually arena exhaustion from many/long label names or values, or a nil/invalid label entry in ts.Labels.

Common situations: Extremely long label names/values from instrumentation; batches near capnp size limits; corrupted or empty label sets produced upstream.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• set tenant
• new labels
• marshal samples
• marshal histograms
• marshal exemplars

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/0daaaa23c5f707e0. Report an issue: GitHub.

Appendix: source

Thrown at pkg/receive/writecapnp/marshal.go:84

func BuildInto(wr *TimeSeriesTenantTuple, tenant string, tsreq []prompb.TimeSeries, builder *symboltable.Builder) error {
	if err := wr.SetTenant(tenant); err != nil {
		return errors.Wrap(err, "set tenant")
	}

	series, err := wr.NewTimeSeries(int32(len(tsreq)))
	if err != nil {
		return err
	}
	for i, ts := range tsreq {
		tsc := series.At(i)

		lblsc, err := tsc.NewLabels(int32(len(ts.Labels)))
		if err != nil {
			return errors.Wrap(err, "new labels")
		}
		if err := marshalLabels(lblsc, ts.Labels, builder); err != nil {
			return errors.Wrap(err, "marshal labels")
		}
		if err := marshalSamples(tsc, ts.Samples); err != nil {
			return errors.Wrap(err, "marshal samples")
		}
		if err := marshalHistograms(tsc, ts.Histograms); err != nil {
			return errors.Wrap(err, "marshal histograms")
		}
		if err := marshalExemplars(tsc, ts.Exemplars, builder); err != nil {
			return errors.Wrap(err, "marshal exemplars")
		}
	}

	return nil
}

func BuildIntoSingleTenantWriteRequest(wr WriteRequest, tenant string, tsreq []prompb.TimeSeries) error {
	if err := wr.SetTenant(tenant); err != nil {
		return errors.Wrap(err, "set tenant")

View on GitHub (pinned to 35b8b99117)