thanos-io/thanos · warning
block has no meta file
Error message
block %s has no meta file
What it means
After listing the bucket, fetchMetadata classifies blocks flagged as partial by the lister and records per-block errors "block %s has no meta file", incrementing resp.noMetas. This is a per-block (not fatal) error meaning a block directory exists in the bucket but its meta.json is absent — typically because the block is still being uploaded (partial block).
Solutions
- If the block is still being uploaded, wait for the upload to finish and re-sync — partial blocks resolve on their own.
- Delete the leftover partial block directory from the bucket if its uploader crashed.
- Check the uploader/compactor logs for failed uploads and re-run the upload.
- Enable IgnoreBlockList or block list filtering to skip known-partial blocks during sync if desired.
Example fix
// before: leftover partial block in bucket with no meta.json // s3://my-bucket/01ARZ3NDEKTSV4RRFFQ69G5FAV/ (only index + chunks, no meta.json) // after: remove the partial block aws s3 rm s3://my-bucket/01ARZ3NDEKTSV4RRFFQ69G5FAV --recursive
Defensive patterns
Strategy: fallback
Validate before calling
// Before treating as fatal: list blocks and check for meta.json presence
_, partial, err := lister.GetActiveAndPartialBlockIDs(ctx, ch)
if err == nil && len(partial) > 0 {
// exclude partial blocks or wait for uploads to finish
} Try / catch
metas, err := fetcher.FetchMetadata(ctx, filtered)
if err != nil {
if strings.Contains(err.Error(), "has no meta file") {
// per-block warning: log the block ULIDs, skip them, retry sync later
} else {
return err
}
} Prevention
- Keep a single uploader per bucket or use ignore-blocks patterns for in-progress blocks.
- Clean up partial block dirs left by crashed uploads before running sync.
- Watch the Synced{state=no-meta} metric; persistent values indicate stuck partial uploads.
- Let block uploaders finish (upload meta.json last, atomically) before syncing.
When it happens
Trigger: fetchMetadata encounters a block ULID in partialBlocks (GetActiveAndPartialBlockIDs returned isPartial=true): the block dir exists but meta.json is missing/not yet uploaded.
Common situations: Concurrent upload in progress (uploader writes dir and index before meta.json); crashed/interrupted upload leaving an empty block dir; manually deleted meta.json; another process mid-upload during sync.
Understand the failure class
Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.
Related errors
- meta.json unmarshal
- sync before first pass of downsampling
- sync before second pass of downsampling
- sync before retention
- create meta fetcher
AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07).
Data as JSON: /api/errors/bbfd195327e76acc.
Report an issue: GitHub.
Appendix: source
Thrown at pkg/block/fetcher.go:599
}
var partialBlocks map[ulid.ULID]bool
var err error
// Workers scheduled, distribute blocks.
eg.Go(func() error {
defer close(activeBlocksCh)
partialBlocks, err = f.blockIDsLister.GetActiveAndPartialBlockIDs(ctx, activeBlocksCh)
return err
})
if err := eg.Wait(); err != nil {
return nil, errors.Wrap(err, "BaseFetcher: iter bucket")
}
mtx.Lock()
for blockULID, isPartial := range partialBlocks {
if isPartial {
resp.partial[blockULID] = errors.Errorf("block %s has no meta file", blockULID)
resp.noMetas++
}
}
mtx.Unlock()
if len(resp.metaErrs) > 0 {
return resp, nil
}
// Only for complete view of blocks update the cache.
cached := &sync.Map{}
for id, m := range resp.metas {
cached.Store(id, m)
}
modifiedTimestamps := make(map[ulid.ULID]time.Time, len(resp.modifiedTimestamps))
maps.Copy(modifiedTimestamps, resp.modifiedTimestamps)View on GitHub (pinned to 35b8b99117)