thanos-io/thanos · error

garbage

Error message

garbage

What it means

Wraps sy.GarbageCollect during the compactor's initial garbage-collection pass: removing source blocks whose compaction result is already durable failed (bucket delete errors or metadata mismatches). The stale blocks survive and GC is retried on the next cycle.

Solutions

1. Check bucket permissions for marking/deleting
2. Inspect wrapped errors for failing blocks
3. Re-run the compactor; GC retries

When it happens

Trigger: Thrown at pkg/compact/compact.go:1582 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/6114037ad90227b1. Report an issue: GitHub.

Appendix: source

Thrown at pkg/compact/compact.go:1582

		// Blocks that were compacted are garbage collected after each Compaction.
		// However if compactor crashes we need to resolve those on startup.
		level.Info(c.logger).Log("msg", "start initial sync of metas")
		if err := c.sy.SyncMetas(ctx); err != nil {
			return errors.Wrap(err, "sync")
		}

		var ignoreBlocks map[ulid.ULID]struct{}
		if c.blocksCleaner != nil {
			deletedBlocks, err := c.blocksCleaner.DeleteMarkedBlocks(ctx)
			if err != nil {
				return errors.Wrap(err, "cleaning marked blocks")
			}
			ignoreBlocks = deletedBlocks
		}

		level.Info(c.logger).Log("msg", "start of initial garbage collection")
		if err := c.sy.GarbageCollect(ctx, ignoreBlocks); err != nil {
			return errors.Wrap(err, "garbage")
		}

		groups, err := c.grouper.Groups(c.sy.Metas())
		if err != nil {
			return errors.Wrap(err, "build compaction groups")
		}

		ignoreDirs := []string{}
		for _, gr := range groups {
			for _, grID := range gr.IDs() {
				ignoreDirs = append(ignoreDirs, filepath.Join(gr.Key(), grID.String()))
			}
		}

		if err := runutil.DeleteAll(c.compactDir, ignoreDirs...); err != nil {
			level.Warn(c.logger).Log("msg", "failed deleting non-compaction group directories/files, some disk space usage might have leaked. Continuing", "err", err, "dir", c.compactDir)
		}

View on GitHub (pinned to 35b8b99117)