thanos-io/thanos · error

get appender

Error message

get appender

What it means

Error from Writer.Write when s.Appender(ctx) fails for any reason other than tsdb.ErrNotReady (which is returned unwrapped). It wraps the underlying storage error encountered while opening an appender for the tenant's TSDB.

Solutions

1. Inspect the wrapped cause; if it is context.Canceled/deadline, check upstream timeouts and receiver load.
2. If tsdb.ErrNotReady, note that code path returns it unwrapped - so a wrapped error here means a different storage problem; check receiver logs for head corruption.
3. Restart/recover the receive pod if the TSDB head is in a bad state; restore from backup if corruption persists.
4. Ensure the receiver has finished startup (TSDB ready) before routing traffic.

Example fix

// caller handling
if err := writer.Write(ctx, tenant, req); err != nil {
    if errors.Is(err, context.DeadlineExceeded) {
        // retry with backoff
    } else { /* surface to sender as 500 */ }
}

if err := writer.Write(ctx, tenant, series); err != nil {
    if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
        // retry with backoff and a fresh context
    }
    // otherwise surface as 500 to sender
}

When it happens

Trigger: TenantAppendable succeeded but the tenant TSDB's Appender(ctx) call returned a non-ErrNotReady error - e.g. head block corrupted, storage closed, or context cancelled during appender creation.

Common situations: TSDB shutting down or reloading while writes arrive; corrupted head data on disk; request context cancelled by an upstream timeout; OOM-related head failures.

Related errors

• get tenant appendable
• commit samples
• start remote write agent db
• open TSDB
• add series

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/eb4bcaff9916cc57. Report an issue: GitHub.

Appendix: source

Thrown at pkg/receive/writer.go:87

		multiTSDB: multiTSDB,
		opts:      opts,
	}
}

func (r *Writer) Write(ctx context.Context, tenantID string, wreq []prompb.TimeSeries) error {
	tLogger := log.With(r.logger, "tenant", tenantID)

	s, err := r.multiTSDB.TenantAppendable(tenantID)
	if err != nil {
		return errors.Wrap(err, "get tenant appendable")
	}

	app, err := s.Appender(ctx)
	if err == tsdb.ErrNotReady {
		return err
	}
	if err != nil {
		return errors.Wrap(err, "get appender")
	}
	getRef := app.(storage.GetRef)
	var (
		ref          storage.SeriesRef
		errorTracker writeErrorTracker
	)
	app = &ReceiveAppender{
		tLogger:        tLogger,
		tooFarInFuture: r.opts.TooFarInFutureTimeWindow,
		Appender:       app,
	}

	for _, t := range wreq {
		// Check if time series labels are valid. If not, skip the time series
		// and report the error.
		if err := labelpb.ValidateLabels(t.Labels); err != nil {
			lset := &labelpb.ZLabelSet{Labels: t.Labels}
			errorTracker.addLabelsError(err, lset, tLogger)

View on GitHub (pinned to 35b8b99117)