{"record":{"id":"d18dad81da57e8c4","repo":"thanos-io/thanos","slug":"meta-json-corrupted","errorCode":null,"errorMessage":"meta.json corrupted","messagePattern":"meta\\.json corrupted","errorType":"error_code","errorClass":"ErrorSyncMetaCorrupted","httpStatus":null,"severity":"error","filePath":"pkg/block/fetcher.go","lineNumber":432,"sourceCode":"\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn b.NewMetaFetcherWithMetrics(fetcherMetrics, filters), nil\n}\n\n// NewMetaFetcher transforms BaseFetcher into actually usable *MetaFetcher.\nfunc (f *BaseFetcher) NewMetaFetcher(reg prometheus.Registerer, filters []MetadataFilter, logTags ...any) *MetaFetcher {\n\treturn f.NewMetaFetcherWithMetrics(NewFetcherMetrics(reg, nil, nil), filters, logTags...)\n}\n\n// NewMetaFetcherWithMetrics transforms BaseFetcher into actually usable *MetaFetcher.\nfunc (f *BaseFetcher) NewMetaFetcherWithMetrics(fetcherMetrics *FetcherMetrics, filters []MetadataFilter, logTags ...any) *MetaFetcher {\n\treturn &MetaFetcher{metrics: fetcherMetrics, wrapped: f, filters: filters, logger: log.With(f.logger, logTags...)}\n}\n\nvar (\n\tErrorSyncMetaNotFound  = errors.New(\"meta.json not found\")\n\tErrorSyncMetaCorrupted = errors.New(\"meta.json corrupted\")\n)\n\nfunc (f *BaseFetcher) metaUpdated(id ulid.ULID, modified time.Time) bool {\n\tif f.modifiedTimestamps[id].IsZero() {\n\t\treturn false\n\t}\n\treturn !f.modifiedTimestamps[id].Equal(modified)\n}\n\nfunc (f *BaseFetcher) bustCacheForID(id ulid.ULID) {\n\tf.cacheBusts.Inc()\n\n\tf.cached.Delete(id)\n\tif err := os.RemoveAll(filepath.Join(f.cacheDir, id.String())); err != nil {\n\t\tlevel.Warn(f.logger).Log(\"msg\", \"failed to remove cached meta.json dir\", \"dir\", filepath.Join(f.cacheDir, id.String()), \"err\", err)\n\t}\n}\n","sourceCodeStart":414,"sourceCodeEnd":450,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/block/fetcher.go#L414-L450","documentation":"ErrorSyncMetaCorrupted is returned by BaseFetcher.loadMeta when meta.json was read successfully but json.Unmarshal could not parse it into metadata.Meta (or filtered fields failed validation earlier). The sentinel wraps the unmarshal error so callers can distinguish corrupt metadata from transient storage errors. Corrupt blocks are skipped and counted in resp.corruptedMetas.","triggerScenarios":"loadMeta: json.Unmarshal(metaContent, m) fails after io.ReadAll succeeded, wrapping ErrorSyncMetaCorrupted with the metaFile path and unmarshal error. FetchBlocks matches errors.Is(err, ErrorSyncMetaCorrupted) and increments resp.corruptedMetas.","commonSituations":"Partial/truncated uploads to object storage (network interruption during block upload); manually edited or hand-crafted meta.json with wrong JSON types (e.g. string where int expected); empty meta.json files from failed uploads; byte corruption in storage.","solutions":["Re-upload or repair the block: download meta.json, fix its JSON (correct types for ULID, minTime/maxTime as int64, version field), and re-upload.","Delete the corrupt block from the bucket (or use thanos tools bucket verify/repair) so syncers stop flagging it.","Check the uploader that wrote the block: ensure meta.json is uploaded last and atomically after all block files.","Inspect resp.corruptedMetas logs to identify the exact block IDs and inspect those meta.json files."],"exampleFix":"// before: hand-edited meta.json with wrong types\n// {\"ulid\": \"01ARZ3...\", \"minTime\": \"1600000000000\"}\n\n// after: correct types per metadata.Meta schema\n// {\"ulid\": \"01ARZ3...\", \"minTime\": 1600000000000, \"maxTime\": 1600003600000, \"version\": 1}","handlingStrategy":"validation","validationCode":"content, err := bkt.Get(ctx, metaPath)\nif err != nil { return err }\nvar m metadata.Meta\nif err := json.Unmarshal(content, &m); err != nil {\n    // corrupt: repair or delete the block before syncing\n}\nif m.ULID.Compare(id) != 0 || m.ThanOSVersion == 0 {\n    // schema mismatch: flag the block\n}","typeGuard":"func isMetaCorrupted(err error) bool {\n    return errors.Is(err, block.ErrorSyncMetaCorrupted)\n}","tryCatchPattern":"if errors.Is(err, block.ErrorSyncMetaCorrupted) {\n    var wrapped *errors.wrapError // extract path via errors.Wrapf message\n    log.Warn(\"corrupt meta.json; excluding block\", \"err\", err)\n    // report the block ID to an operator queue for repair\n}","preventionTips":["Always upload meta.json last and atomically after all chunk/index files","Never hand-edit meta.json; regenerate it with thanos tools bucket tooling","Validate uploads with 'thanos tools bucket verify'","Alert on resp.corruptedMetas > 0 — corrupt blocks never become queryable"],"tags":["json","corruption","thanos","block-sync","metadata"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}