thanos-io/thanos · error

could not group metadata for retention

Error message

could not group metadata for retention

What it means

The same grouper.Groups(metas) call is re-run for the retention calculator; a failure here is wrapped as 'could not group metadata for retention'. Like the compaction grouping error, it indicates incompatible or duplicate blocks within a group (ErrGroupPermission / ErrGroupNotFound).

Solutions

1. Run 'thanos tools bucket verify' and 'repair --mark-logical-deleted' to deduplicate overlapping blocks.
2. Validate meta.json resolution values in the bucket are one of 0/5m/1h.
3. Guarantee a single compactor instance per bucket.
4. Fix external_labels on the Prometheus side so groups are unambiguous.

Example fix

// before
// overlapping blocks fail retention grouping
// after
thanos tools bucket verify --objstore.bucket=thanos
thanos tools bucket repair --objstore.bucket=thanos --mark-logical-deleted

// Ensure resolutions in metadata are valid before grouping
for _, m := range metas {
    switch m.Thanos.Resolution {
    case 0, 5*time.Minute, time.Hour:
    default:
        log.Printf("invalid resolution %d in block %s", m.Thanos.Resolution, m.ULID)
    }
}

retGroups, err := grouper.Groups(metas)
if err != nil {
    log.Errorf("could not group metadata for retention: %v", err)
    alertDuplicateBlocks(err) // verify/repair bucket
    return nil
}

When it happens

Trigger: grouper.Groups(metas) fails while building groups for retention calculation — duplicate blocks with the same external labels/resolution, or blocks referencing unknown resolution values.

Common situations: Duplicate blocks from concurrent compactors, backfilled blocks with clashing labels, or meta.json files with unexpected 'resolution' values (e.g. hand-edited metadata).

Related errors

• raw resolution must be higher than the minimum block size…
• 5m resolution retention must be higher than the minimum…
• sync before retention
• retention failed
• could not group metadata for compaction

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/db80f656e4f30035. Report an issue: GitHub.

Appendix: source

Thrown at cmd/thanos/compact.go:675

							return nil
						}

						return errors.Wrapf(err, "could not sync metas")
					}

					metas := sy.Metas()
					groups, err := grouper.Groups(metas)
					if err != nil {
						return errors.Wrapf(err, "could not group metadata for compaction")
					}

					if err = ps.ProgressCalculate(ctx, groups); err != nil {
						return errors.Wrapf(err, "could not calculate compaction progress")
					}

					retGroups, err := grouper.Groups(metas)
					if err != nil {
						return errors.Wrapf(err, "could not group metadata for retention")
					}

					if err = rs.ProgressCalculate(ctx, retGroups); err != nil {
						return errors.Wrapf(err, "could not calculate retention progress")
					}

					if !conf.disableDownsampling {
						groups, err = grouper.Groups(metas)
						if err != nil {
							return errors.Wrapf(err, "could not group metadata into downsample groups")
						}
						if err := ds.ProgressCalculate(ctx, groups); err != nil {
							return errors.Wrapf(err, "could not calculate downsampling progress")
						}
					}

					return nil
				})

View on GitHub (pinned to 35b8b99117)