thanos-io/thanos · error

cleaning marked blocks

Error message

cleaning marked blocks

What it means

Wraps blocksCleaner.DeleteMarkedBlocks in the compactor startup path: deleting blocks that carry deletion marks from object storage failed. Marked blocks remain, which normally only delays cleanup rather than corrupting state, so the error surfaces for operator attention.

Solutions

1. Check bucket delete permissions
2. Inspect the wrapped error for the failing block
3. Retry next compaction cycle

When it happens

Trigger: Thrown at pkg/compact/compact.go:1575 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/d8953bc8ef6b4d81. Report an issue: GitHub.

Appendix: source

Thrown at pkg/compact/compact.go:1575

					}
					errChan <- errors.Wrapf(err, "group %s", g.Key())
					return
				}
			})
		}

		// Blocks that were compacted are garbage collected after each Compaction.
		// However if compactor crashes we need to resolve those on startup.
		level.Info(c.logger).Log("msg", "start initial sync of metas")
		if err := c.sy.SyncMetas(ctx); err != nil {
			return errors.Wrap(err, "sync")
		}

		var ignoreBlocks map[ulid.ULID]struct{}
		if c.blocksCleaner != nil {
			deletedBlocks, err := c.blocksCleaner.DeleteMarkedBlocks(ctx)
			if err != nil {
				return errors.Wrap(err, "cleaning marked blocks")
			}
			ignoreBlocks = deletedBlocks
		}

		level.Info(c.logger).Log("msg", "start of initial garbage collection")
		if err := c.sy.GarbageCollect(ctx, ignoreBlocks); err != nil {
			return errors.Wrap(err, "garbage")
		}

		groups, err := c.grouper.Groups(c.sy.Metas())
		if err != nil {
			return errors.Wrap(err, "build compaction groups")
		}

		ignoreDirs := []string{}
		for _, gr := range groups {
			for _, grID := range gr.IDs() {
				ignoreDirs = append(ignoreDirs, filepath.Join(gr.Key(), grID.String()))

View on GitHub (pinned to 35b8b99117)