thanos-io/thanos · error

errObjNotFound

errObjNotFound

Error message

object not found

What it means

CachingBucket.Get returns errObjNotFound when the cached 'exists' marker for an object is false, meaning the cache has a recent negative answer that the object does not exist in the bucket. IsObjNotFoundErr treats this sentinel the same as the underlying bucket's own not-found errors, so callers can handle both uniformly as object-not-found.

Solutions

1. Use cb.IsObjNotFoundErr(err) to branch on not-found and skip the object instead of treating it as a hard failure.
2. Invalidate the cached 'exists' entry (delete the existence cache key) if the object may have been uploaded since.
3. Lower the existence cache TTL (ExistsTTL / eventual-upload TTL in CachingBucketConfig) so false negatives expire sooner.
4. Re-check the source bucket directly if stale negative caching is suspected.

Example fix

// before
rc, err := cb.Get(ctx, name)
if err != nil {
    return err
}
// after
rc, err := cb.Get(ctx, name)
if cb.IsObjNotFoundErr(err) {
    return nil // object genuinely absent (or stale negative cache)
}
if err != nil {
    return errors.Wrapf(err, "get %s", name)
}

// Pre-check existence explicitly and accept its answer as TTL-cached
exists, err := cb.Exists(ctx, name)
if err == nil && !exists {
    return nil, nil
}

func isObjNotFound(err error, cb *cache.CachingBucket) bool {
    return cb.IsObjNotFoundErr(err)
}

rc, err := cb.Get(ctx, name)
if err != nil {
    if cb.IsObjNotFoundErr(err) {
        return nil, ErrObjectGone // handle uniformly as not-found
    }
    return errors.Wrapf(err, "get %s", name)
}

When it happens

Trigger: Calling CachingBucket.Get (or Exists) for an object that a previous Get/Exists call cached as nonexistent (existence TTL not yet expired); the wrapped error is then matched via IsObjNotFoundErr.

Common situations: Reading objects deleted shortly after being cached as missing (or cached as present then deleted); racing a compaction/upload pipeline; block deletion while a query still references it.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• failed to get object attributes
• ErrInvalidBucketCacheKeyFormat
• ErrInvalidBucketCacheKeyVerb
• ErrParseKeyInt
• bucket is nil

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/8876c8f3dc2046d6. Report an issue: GitHub.

Appendix: source

Thrown at pkg/store/cache/caching_bucket.go:35

	"github.com/pkg/errors"
	"github.com/prometheus/client_golang/prometheus"
	"github.com/prometheus/client_golang/prometheus/promauto"
	"golang.org/x/sync/errgroup"

	"github.com/thanos-io/objstore"

	"github.com/thanos-io/thanos/pkg/cache"
	"github.com/thanos-io/thanos/pkg/runutil"
	"github.com/thanos-io/thanos/pkg/store/cache/cachekey"
)

const (
	originCache  = "cache"
	originBucket = "bucket"
)

var (
	errObjNotFound = errors.Errorf("object not found")
)

// CachingBucket implementation that provides some caching features, based on passed configuration.
type CachingBucket struct {
	objstore.Bucket

	cfg    *cache.CachingBucketConfig
	logger log.Logger

	requestedGetRangeBytes *prometheus.CounterVec
	fetchedGetRangeBytes   *prometheus.CounterVec
	refetchedGetRangeBytes *prometheus.CounterVec

	operationConfigs  map[string][]*cache.OperationConfig
	operationRequests *prometheus.CounterVec
	operationHits     *prometheus.CounterVec
}

View on GitHub (pinned to 35b8b99117)