{"record":{"id":"3f142cfb71ae6d92","repo":"thanos-io/thanos","slug":"read-parquet-metadata-files","errorCode":null,"errorMessage":"read parquet metadata files","messagePattern":"read parquet metadata files","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/block/fetcher.go","lineNumber":1142,"sourceCode":"\n\t\treturn f.bkt.Iter(ctx, \"\", func(name string) error {\n\t\t\tif path.Base(name) == parquetMetaFileName {\n\t\t\t\tselect {\n\t\t\t\tcase ch <- name:\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\treturn ctx.Err()\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil\n\t\t}, objstore.WithRecursiveIter())\n\t})\n\n\tif err := eg.Wait(); err != nil {\n\t\treturn errors.Wrap(err, \"iterate bucket for parquet metadata\")\n\t}\n\n\tif len(merrs) > 0 {\n\t\treturn errors.Wrap(merrs.Err(), \"read parquet metadata files\")\n\t}\n\n\t// NOTE(GiedriusS): we need to be extra careful here because Parquet converter trims chunks exactly to the day's range.\n\t// The input to the Parquet converter might not be aligned exactly so we NEED to have some overlap.\n\t// So, only delete metas if the migrated blocks cover each day *fully*.\n\tfor id := range filterMigratedBlocksByDayCoverage(allMigratedBlocks, metas) {\n\t\tdelete(metas, id)\n\t\tsynced.WithLabelValues(\"parquet-converted\").Inc()\n\t}\n\n\treturn nil\n}\n\nfunc filterMigratedBlocksByDayCoverage(migratedBlocks map[ulid.ULID]struct{}, metas map[ulid.ULID]*metadata.Meta) map[ulid.ULID]struct{} {\n\ttype blockRange struct {\n\t\tid               ulid.ULID\n\t\tminTime, maxTime int64\n\t}","sourceCodeStart":1124,"sourceCodeEnd":1160,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/block/fetcher.go#L1124-L1160","documentation":"Returned when one or more of the concurrently executed parquet metadata file reads failed (merrs aggregated). It wraps merrs.Err() with 'read parquet metadata files', distinct from the bucket iteration error above it.","triggerScenarios":"Any readMigratedBlocksFromParquetMetadata failure for a listed parquet metadata file — Get errors, io.ReadAll errors, easyproto field parse failures — aggregated by the errgroup workers.","commonSituations":"Objects deleted between listing and reading; truncated parquet metadata from an interrupted parquet conversion; permission or throttling errors on Get.","solutions":["Check the inner aggregated errors for the specific file paths and root causes","Delete or repair corrupt/truncated parquet metadata files, or re-run the parquet converter","Verify read permissions on the parquet metadata prefix","Retry if the underlying errors are transient (throttling, network)"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"if err := refreshMigratedBlocks(ctx); err != nil {\n    var errs []error\n    if errors.As(err, &errs) {\n        for _, e := range errs { log.Printf(\"parquet meta read: %v\", e) }\n    }\n    // keep previous migrated-blocks set rather than wiping it\n}","preventionTips":["Ensure the converter finishes writes before readers scan","Set correct GetObject IAM permissions","Handle throttling with reduced concurrency/backoff","Delete corrupt metadata files instead of failing every scan"],"tags":["object-storage","parquet","aggregated-errors"],"backgroundTag":"file-read-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}