thanos-io/thanos · error

unexpected deletion-mark file version

Error message

unexpected deletion-mark file version %d, expected %d

What it means

ReadMarker verifies that a deletion-mark file's Version field equals DeletionMarkVersion1. This error is raised when the deletion marker JSON parses but its version is not the supported version 1, indicating the marker was written by an incompatible format. Because deletion marks gate whether a block's data can be removed, M3DB refuses to trust a marker with an unknown version.

Solutions

1. Inspect the Version field in the deletion-mark file and determine which M3DB version wrote it.
2. Run a binary that supports that marker version (upgrade/downgrade the node accordingly).
3. If the deletion was already applied or is safe to re-issue, delete the malformed deletion-mark and re-run the block deletion via the supported API.
4. Never hand-edit deletion markers; use the m3ctl/deletion tooling so version fields are set correctly.

Example fix

// before: hand-edited deletion-mark {"Version": 99, ...} -> error

// after: re-issue deletion through the supported path
client.DeleteBlocks(namespace, blockIDs) // writes DeletionMark{Version: DeletionMarkVersion1}

var m struct{ Version int `json:"version"` }
if json.Unmarshal(data, &m) == nil && m.Version != metadata.DeletionMarkVersion1 {
    // do not trust this deletion mark
}

func isKnownDeletionMarkVersion(v int) bool { return v == metadata.DeletionMarkVersion1 }

if err := metadata.ReadMarker(path, marker); err != nil {
    if strings.Contains(err.Error(), "unexpected deletion-mark file version") {
        // refuse deletion flow; re-issue deletion via supported API
        return fmt.Errorf("deletion-mark version skew: %w", err)
    }
    return err
}

When it happens

Trigger: Calling ReadMarker on a deletion-mark file whose decoded Version differs from DeletionMarkVersion1 — e.g. a deletion mark written by a newer M3DB release, or a file whose version field was edited or corrupted after a crash.

Common situations: Rolling upgrade with mixed node versions; restoring an old snapshot under a newer binary (or vice versa); operator manually editing deletion markers to force block deletion; partial disk writes corrupting the version field.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• unexpected no-compact-mark file version
• unexpected no-downsample-mark file version
• unexpected meta file
• not supported index file version
• unexpected meta file version

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/004fca837f41d96e. Report an issue: GitHub.

Appendix: source

Thrown at pkg/block/metadata/markers.go:149

	if err != nil {
		return errors.Wrapf(err, "read file: %s", markerFile)
	}

	if err := json.Unmarshal(metaContent, marker); err != nil {
		return errors.Wrapf(ErrorUnmarshalMarker, "file: %s; err: %v", markerFile, err.Error())
	}
	switch marker.markerFilename() {
	case NoCompactMarkFilename:
		if version := marker.(*NoCompactMark).Version; version != NoCompactMarkVersion1 {
			return errors.Errorf("unexpected no-compact-mark file version %d, expected %d", version, NoCompactMarkVersion1)
		}
	case NoDownsampleMarkFilename:
		if version := marker.(*NoDownsampleMark).Version; version != NoDownsampleMarkVersion1 {
			return errors.Errorf("unexpected no-downsample-mark file version %d, expected %d", version, NoDownsampleMarkVersion1)
		}
	case DeletionMarkFilename:
		if version := marker.(*DeletionMark).Version; version != DeletionMarkVersion1 {
			return errors.Errorf("unexpected deletion-mark file version %d, expected %d", version, DeletionMarkVersion1)
		}
	}
	return nil
}

View on GitHub (pinned to 35b8b99117)