thanos-io/thanos · error

get appender

Error message

get appender

What it means

This error wraps a failure from the TSDB's Appender(ctx) call inside the Cap'n Proto receive writer. Thanos Receive first obtains a head Appender to buffer incoming samples; if the local TSDB cannot provide one (for any reason other than ErrNotReady, which is returned unwrapped for retry), the write fails with 'get appender'. It indicates the local storage head is not accepting appends right now.

Solutions

1. Check the wrapped cause in the error chain for the underlying Appender failure
2. Verify the TSDB data directory is writable and not corrupted
3. Ensure Receive is not being shut down while still receiving traffic (check graceful termination order)
4. If tsdb.ErrNotReady is returned instead, wait for head readiness before retrying

Example fix

// before
app, err := s.Appender(ctx)
if err != nil { log.Error(err) }
// after
app, err := s.Appender(ctx)
if errors.Is(err, tsdb.ErrNotReady) {
	time.Sleep(backoff); return retry
}
if err != nil {
	return errors.Wrap(err, "get appender")
}

// no pre-call check possible; gate on readiness
if s == nil || s.BlockStore == nil {
	return errors.New("storage not initialized")
}

app, err := w.Write(ctx, req)
if errors.Is(err, tsdb.ErrNotReady) {
	// backoff and retry same batch
} else if err != nil {
	// inspect wrapped cause via errors.Cause before giving up
}

When it happens

Trigger: Calling CapNProtoWriter.Write when the embedded TSDB head is shutting down, corrupted, or its Appender() call fails for reasons other than tsdb.ErrNotReady (e.g. storage closed, OOM during head init).

Common situations: Receive pod shutting down while traffic still arrives; TSDB head compaction/startup in progress with a non-retryable failure; storage backend errors during appender acquisition; misconfigured storage paths on disk.

Related errors

• start remote write agent db
• open TSDB
• open index reader
• open chunk reader
• open tombstone reader

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/7855b48dd479ced7. Report an issue: GitHub.

Appendix: source

Thrown at pkg/receive/capnproto_writer.go:56

		multiTSDB: multiTSDB,
		opts:      opts,
	}
}

func (r *CapNProtoWriter) Write(ctx context.Context, wreq *writecapnp.Request) error {
	tLogger := log.With(r.logger, "tenant", wreq.Tenant)

	s, err := r.multiTSDB.TenantAppendable(wreq.Tenant)
	if err != nil {
		return errors.Wrap(err, "get tenant appendable")
	}

	app, err := s.Appender(ctx)
	if err == tsdb.ErrNotReady {
		return err
	}
	if err != nil {
		return errors.Wrap(err, "get appender")
	}
	getRef := app.(storage.GetRef)
	var (
		ref          storage.SeriesRef
		errorTracker = &writeErrorTracker{}
	)
	app = &ReceiveAppender{
		tLogger:        tLogger,
		tooFarInFuture: r.opts.TooFarInFutureTimeWindow,
		Appender:       app,
	}

	var (
		series  writecapnp.Series
		builder labels.ScratchBuilder
	)
	for wreq.Next() {
		if err := wreq.At(&series); err != nil {

View on GitHub (pinned to 35b8b99117)