{"record":{"id":"6da31ef9100dd697","repo":"thanos-io/thanos","slug":"read-meta-file-v","errorCode":null,"errorMessage":"read meta file: %v","messagePattern":"read meta file: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/block/fetcher.go","lineNumber":491,"sourceCode":"\t\t\t\tlevel.Warn(f.logger).Log(\"msg\", \"best effort remove of cached dir failed; ignoring\", \"dir\", cachedBlockDir, \"err\", err)\n\t\t\t}\n\t\t}\n\t}\n\n\tr, err := f.bkt.ReaderWithExpectedErrs(f.bkt.IsObjNotFoundErr).Get(ctx, metaFile)\n\tif f.bkt.IsObjNotFoundErr(err) {\n\t\t// Meta.json was deleted between bkt.Exists and here.\n\t\treturn nil, errors.Wrapf(ErrorSyncMetaNotFound, \"%v\", err)\n\t}\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"get meta file: %v\", metaFile)\n\t}\n\n\tdefer runutil.CloseWithLogOnErr(f.logger, r, \"close bkt meta get\")\n\n\tmetaContent, err := io.ReadAll(r)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"read meta file: %v\", metaFile)\n\t}\n\n\tm := &metadata.Meta{}\n\tif err := json.Unmarshal(metaContent, m); err != nil {\n\t\treturn nil, errors.Wrapf(ErrorSyncMetaCorrupted, \"meta.json %v unmarshal: %v\", metaFile, err)\n\t}\n\n\tif m.Version != metadata.TSDBVersion1 {\n\t\treturn nil, errors.Errorf(\"unexpected meta file: %s version: %d\", metaFile, m.Version)\n\t}\n\n\t// Best effort cache in local dir.\n\tif f.cacheDir != \"\" {\n\t\tif err := os.MkdirAll(cachedBlockDir, os.ModePerm); err != nil {\n\t\t\tlevel.Warn(f.logger).Log(\"msg\", \"best effort mkdir of the meta.json block dir failed; ignoring\", \"dir\", cachedBlockDir, \"err\", err)\n\t\t}\n\n\t\tif err := m.WriteToDir(f.logger, cachedBlockDir); err != nil {","sourceCodeStart":473,"sourceCodeEnd":509,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/block/fetcher.go#L473-L509","documentation":"loadMeta wraps an io.ReadAll failure on the opened meta.json reader as errors.Wrapf(err, \"read meta file: %v\", metaFile). The Get succeeded but streaming the object body failed mid-read — typically a dropped connection or checksum/IO error while downloading. The error carries the file path and the underlying cause; the block's metadata cannot be loaded and the sync of that block fails.","triggerScenarios":"loadMeta: io.ReadAll(r) after a successful bkt Get returns err (connection reset during body read, S3 content-length mismatch, disk/proxy failure), wrapped with 'read meta file: <metaFile>'.","commonSituations":"Unstable network or proxy idle timeouts killing long reads; S3/GCS returning truncated bodies; object sizes altered mid-transfer; TLS termination issues; large-scale syncs amplifying transient failures.","solutions":["Retry the sync — this is usually transient; the fetcher will pick the block up on the next iteration.","Check network stability/proxy idle timeouts between Thanos and the object store; raise proxy read timeouts.","Verify the object in the bucket is intact (compare content-length/checksum); re-upload if truncated.","If persistent, inspect the objstore client logs for the underlying cause and adjust retry/backoff settings."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// pre-flight: confirm object is readable end-to-end\nrc, err := bkt.Get(ctx, metaPath)\nif err == nil {\n    _, err = io.Copy(io.Discard, rc) // surfaces read errors early\n    rc.Close()\n}","typeGuard":"func isReadMetaFailure(err error) bool {\n    return err != nil && strings.Contains(err.Error(), \"read meta file:\")\n}","tryCatchPattern":"m, err := loadMetaFn(ctx)\nif err != nil && strings.Contains(err.Error(), \"read meta file:\") {\n    // transient body-read failure: retry with exponential backoff\n    return backoff.Retry(ctx, loadMetaFn, backoff.WithMaxRetries(3))\n}","preventionTips":["Raise proxy/load-balancer idle timeouts for long object reads","Ensure stable network paths between Thanos and object storage","Verify object integrity (ETag/checksum) if truncation recurs","Enable client-level retry on body read errors in the objstore config"],"tags":["object-storage","network","io","thanos","retryable"],"backgroundTag":"file-read-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}