thanos-io/thanos · error

new symbols

Error message

new symbols

What it means

Wrap of the capnp error from wr.NewSymbols(), which allocates the symbol-list field on the WriteRequest message in BuildIntoSingleTenantWriteRequest. It means capnp could not create the symbols list on the message; the raw capnp cause is wrapped underneath.

Solutions

1. Always create the WriteRequest from a fresh capnp.NewMessage(SingleSegment(nil)) before calling this function; never reuse a marshaled message.
2. Reduce request size so allocation fits the arena.
3. Check the wrapped capnp error for the precise failure.
4. Upgrade capnp library if hitting a known allocation bug.

Example fix

// before: reusing wr across calls
wr := previouslyMarshaledWriteRequest
BuildIntoSingleTenantWriteRequest(wr, tenant, series)
// after: fresh message each time
_, seg, _ := capnp.NewMessage(capnp.SingleSegment(nil))
wr, _ := NewRootWriteRequest(seg)
BuildIntoSingleTenantWriteRequest(wr, tenant, series)

if err := BuildIntoSingleTenantWriteRequest(wr, tenant, series); err != nil {
    if strings.Contains(err.Error(), "new symbols") {
        // rebuild wr from a fresh capnp.NewMessage arena and retry once
    }
}

When it happens

Trigger: BuildIntoSingleTenantWriteRequest reaching the symbols step after all series were marshaled, and the capnp message cannot allocate the new symbols list (arena exhausted, message finalized/already marshaled).

Common situations: Reusing a WriteRequest whose message was already marshaled or came from a read-only view; extremely large write requests consuming the whole segment before the symbols step.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• marshal symbols
• set tenant
• new labels
• marshal labels
• marshal samples

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/75b7d58509a19762. Report an issue: GitHub.

Appendix: source

Thrown at pkg/receive/writecapnp/marshal.go:133

			return errors.Wrap(err, "new labels")
		}
		if err := marshalLabels(lblsc, ts.Labels, builder); err != nil {
			return errors.Wrap(err, "marshal labels")
		}
		if err := marshalSamples(tsc, ts.Samples); err != nil {
			return errors.Wrap(err, "marshal samples")
		}
		if err := marshalHistograms(tsc, ts.Histograms); err != nil {
			return errors.Wrap(err, "marshal histograms")
		}
		if err := marshalExemplars(tsc, ts.Exemplars, builder); err != nil {
			return errors.Wrap(err, "marshal exemplars")
		}
	}

	symbols, err := wr.NewSymbols()
	if err != nil {
		return errors.Wrap(err, "new symbols")
	}
	if err := marshalSymbols(builder, symbols); err != nil {
		return errors.Wrap(err, "marshal symbols")
	}
	return nil
}

func marshalSymbols(builder *symboltable.Builder, symbols Symbols) error {
	offsets, err := symbols.NewOffsets(builder.Len())
	if err != nil {
		return err
	}
	data := make([]byte, builder.SymbolsSize)
	for k, entry := range builder.Table {
		end := entry.Start + uint32(len(k))
		copy(data[entry.Start:end], k)
		offsets.Set(int(entry.Index), end)
	}

View on GitHub (pinned to 35b8b99117)