{"record":{"id":"56c33bf92b7fce72","repo":"thanos-io/thanos","slug":"file-s-err-v","errorCode":null,"errorMessage":"file: %s; err: %v","messagePattern":"file: (.+?); err: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/block/metadata/markers.go","lineNumber":136,"sourceCode":"// ReadMarker reads the given mark file from <dir>/<marker filename>.json in bucket.\nfunc ReadMarker(ctx context.Context, logger log.Logger, bkt objstore.InstrumentedBucketReader, dir string, marker Marker) error {\n\tmarkerFile := path.Join(dir, marker.markerFilename())\n\tr, err := bkt.ReaderWithExpectedErrs(bkt.IsObjNotFoundErr).Get(ctx, markerFile)\n\tif err != nil {\n\t\tif bkt.IsObjNotFoundErr(err) {\n\t\t\treturn ErrorMarkerNotFound\n\t\t}\n\t\treturn errors.Wrapf(err, \"get file: %s\", markerFile)\n\t}\n\tdefer runutil.CloseWithLogOnErr(logger, r, \"close bkt marker reader\")\n\n\tmetaContent, err := io.ReadAll(r)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"read file: %s\", markerFile)\n\t}\n\n\tif err := json.Unmarshal(metaContent, marker); err != nil {\n\t\treturn errors.Wrapf(ErrorUnmarshalMarker, \"file: %s; err: %v\", markerFile, err.Error())\n\t}\n\tswitch marker.markerFilename() {\n\tcase NoCompactMarkFilename:\n\t\tif version := marker.(*NoCompactMark).Version; version != NoCompactMarkVersion1 {\n\t\t\treturn errors.Errorf(\"unexpected no-compact-mark file version %d, expected %d\", version, NoCompactMarkVersion1)\n\t\t}\n\tcase NoDownsampleMarkFilename:\n\t\tif version := marker.(*NoDownsampleMark).Version; version != NoDownsampleMarkVersion1 {\n\t\t\treturn errors.Errorf(\"unexpected no-downsample-mark file version %d, expected %d\", version, NoDownsampleMarkVersion1)\n\t\t}\n\tcase DeletionMarkFilename:\n\t\tif version := marker.(*DeletionMark).Version; version != DeletionMarkVersion1 {\n\t\t\treturn errors.Errorf(\"unexpected deletion-mark file version %d, expected %d\", version, DeletionMarkVersion1)\n\t\t}\n\t}\n\treturn nil\n}\n","sourceCodeStart":118,"sourceCodeEnd":154,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/block/metadata/markers.go#L118-L154","documentation":"ReadMarker in pkg/block/metadata/markers.go reads a marker file (no-compact-mark, no-downsample-mark, deletion-mark) from a block metadata directory and unmarshals its JSON into the marker struct. This error is raised when the marker file exists but its bytes are not valid JSON for the marker type, wrapping ErrorUnmarshalMarker with the file path and the underlying json error. M3DB throws it so operators know a metadata marker file is corrupt or was written by an incompatible format.","triggerScenarios":"Calling ReadMarker on a marker file whose contents cannot be json.Unmarshal-ed into the expected marker struct: truncated file (crash during WriteMarker), manually edited marker file with invalid JSON, empty file, or a file written with a different schema/field types than the marker struct expects.","commonSituations":"Disk full or node crash left a partially written no-compact-mark or deletion-mark file; an operator hand-edited the marker; backup/restore copied a corrupted marker; a version mismatch introduced fields the struct cannot decode (e.g. string where int expected).","solutions":["Inspect the marker file's contents (cat the path shown in the error) to confirm whether the JSON is valid.","Delete the corrupted marker file if the operation it guards (e.g. deletion or compaction skip) can be safely re-applied, then re-run the operation.","Restore the marker file from a backup or replicate it from a healthy node in the cluster.","If this recurs, check disk health and fsync behavior on the node; a failing disk can corrupt metadata writes."],"exampleFix":"// before: blindly retrying ReadMarker on a corrupt file\nerr := metadata.ReadMarker(filePath, marker) // keeps failing\n\n// after: validate then recreate the marker\nif err := metadata.ReadMarker(filePath, marker); err != nil {\n    if errors.Is(err, metadata.ErrorUnmarshalMarker) {\n        os.Remove(filePath) // drop corrupt marker, re-create via WriteMarker\n    }\n}","handlingStrategy":"try-catch","validationCode":"data, err := os.ReadFile(markerFile)\nif err == nil && !json.Valid(data) {\n    // treat as corrupt marker before calling ReadMarker\n}","typeGuard":"func isMarkerJSONValid(path string) bool {\n    b, err := os.ReadFile(path)\n    return err == nil && json.Valid(b)\n}","tryCatchPattern":"if err := metadata.ReadMarker(path, marker); err != nil {\n    if errors.Is(err, metadata.ErrorUnmarshalMarker) {\n        // corrupt marker: quarantine/delete file and recreate\n        os.Remove(path)\n    }\n    return err\n}","preventionTips":["Always write marker files via WriteMarker (atomic write + fsync), never with ad-hoc os.WriteFile.","Do not hand-edit marker files in block metadata directories.","Monitor for disk-full and unclean shutdowns; they leave truncated markers.","When copying namespace data between nodes, verify marker JSON validity after restore."],"tags":["json","metadata","file-corruption","m3db"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}