thanos-io/thanos · error

add block to set

Error message

add block to set

What it means

Wraps bucketBlockSet.add failure after a block was successfully opened: inserting the loaded block into the block set for its labels failed. In practice this happens when the block's time range or metadata fails the set's consistency checks (e.g. duplicate ULID or overlapping-block invariants enforced by the set).

Solutions

1. Run 'thanos tools bucket verify' to detect duplicate or inconsistent blocks in the bucket.
2. Remove duplicate blocks with the same ULID (usually leftover from a failed compactor upload).
3. Ensure only one compactor instance runs per bucket to prevent concurrent duplicate uploads.
4. Retry the block sync; transient races usually resolve on the next sync cycle.

// Detect duplicate blocks before load
seen := map[ulid.ULID]struct{}{}
for _, id := range blockIDs {
	if _, dup := seen[id]; dup {
		log.Warn("duplicate block ULID in bucket", "id", id)
	}
	seen[id] = struct{}{}
}

if err := set.add(b); err != nil {
	logger.Warn("retrying block add after next sync", "block", b.meta.ULID, "err", err)
	_ = b.Close()
	return nil // defer to next sync cycle instead of failing the whole sync
}

When it happens

Trigger: set.add(b) on a newly created bucketBlockSet returns an error, typically because two blocks with the same ULID or inconsistent metadata (time ranges, labels) are added during syncBlocks mapping.

Common situations: Bucket contents changed concurrently between listing and loading (block deleted then re-added), duplicate block instances in the bucket due to a compactor race, corrupted meta.json producing conflicting metadata.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• new bucket block
• close block
• invalid argument: --min-time
• error while parsing config for request logging
• meta.json not found

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/21bd87e38b69a9fb. Report an issue: GitHub.

Appendix: source

Thrown at pkg/store/bucket.go:914

		return errors.Wrap(err, "new bucket block")
	}
	defer func() {
		if err != nil {
			runutil.CloseWithErrCapture(&err, b, "index-header")
		}
	}()

	s.mtx.Lock()
	defer s.mtx.Unlock()

	set, ok := s.blockSets[h]
	if !ok {
		set = newBucketBlockSet(lset)
		s.blockSets[h] = set
	}

	if err = set.add(b); err != nil {
		return errors.Wrap(err, "add block to set")
	}
	s.blocks[b.meta.ULID] = b

	s.metrics.blocksLoaded.Inc()
	s.metrics.lastLoadedBlock.SetToCurrentTime()
	return nil
}

func (s *BucketStore) removeBlock(id ulid.ULID) error {
	s.mtx.Lock()
	b, ok := s.blocks[id]
	if ok {
		lset := labels.FromMap(b.meta.Thanos.Labels)
		s.blockSets[lset.Hash()].remove(id)
		delete(s.blocks, id)
	}
	s.mtx.Unlock()

View on GitHub (pinned to 35b8b99117)