{"record":{"id":"fb49a4686d6c8575","repo":"thanos-io/thanos","slug":"first-pass-of-downsampling-failed","errorCode":null,"errorMessage":"first pass of downsampling failed","messagePattern":"first pass of downsampling failed","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cmd/thanos/compact.go","lineNumber":481,"sourceCode":"\t\t\tfor _, meta := range filteredMetas {\n\t\t\t\tresolutionLabel := meta.Thanos.ResolutionString()\n\t\t\t\tdownsampleMetrics.downsamples.WithLabelValues(resolutionLabel)\n\t\t\t\tdownsampleMetrics.downsampleFailures.WithLabelValues(resolutionLabel)\n\t\t\t}\n\n\t\t\tif err := downsampleBucket(\n\t\t\t\tctx,\n\t\t\t\tlogger,\n\t\t\t\tdownsampleMetrics,\n\t\t\t\tinsBkt,\n\t\t\t\tfilteredMetas,\n\t\t\t\tdownsamplingDir,\n\t\t\t\tconf.downsampleConcurrency,\n\t\t\t\tconf.blockFilesConcurrency,\n\t\t\t\tmetadata.HashFunc(conf.hashFunc),\n\t\t\t\tconf.acceptMalformedIndex,\n\t\t\t); err != nil {\n\t\t\t\treturn errors.Wrap(err, \"first pass of downsampling failed\")\n\t\t\t}\n\n\t\t\tlevel.Info(logger).Log(\"msg\", \"start second pass of downsampling\")\n\t\t\tif err := sy.SyncMetas(ctx); err != nil {\n\t\t\t\treturn errors.Wrap(err, \"sync before second pass of downsampling\")\n\t\t\t}\n\n\t\t\t// Regenerate the filtered list of blocks after the sync,\n\t\t\t// to include the blocks created by the first pass.\n\t\t\tfilteredMetas = sy.Metas()\n\t\t\tnoDownsampleBlocks = noDownsampleMarkerFilter.NoDownsampleMarkedBlocks()\n\t\t\tfor ul := range noDownsampleBlocks {\n\t\t\t\tdelete(filteredMetas, ul)\n\t\t\t}\n\n\t\t\tif err := downsampleBucket(\n\t\t\t\tctx,\n\t\t\t\tlogger,","sourceCodeStart":463,"sourceCodeEnd":499,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/cmd/thanos/compact.go#L463-L499","documentation":"Wrapper error from the first downsampling pass in runCompact (cmd/thanos/compact.go:481) around downsample.DownsampleBucket (or equivalent). An individual block downsampling job failed — the specific block, hash function, or concurrency worker error is wrapped inside.","triggerScenarios":"Downsampling a specific block fails: corrupt or unreadable index/chunks files in the local downsampling dir, unsupported hash function, IO errors writing the downsampled block, concurrent workers exhausted context.","commonSituations":"Disk full on --data-dir volume, malformed blocks flagged by --accept-malformed-index not set, hash function mismatch (e.g. SHA256 vs XXHash) with objects in the bucket.","solutions":["Read the wrapped inner error to identify the failing block and stage","Free disk space or fix volume permissions on the downsampling/data directory","Re-download or delete the corrupt source block (after verifying) so compaction can regenerate it","Match --hash-func with the hash function used when blocks were uploaded","Set --accept-malformed-index only if you understand the malformed index risk"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"if err := downsampleBucket(...); err != nil {\n\tlevel.Error(logger).Log(\"msg\", \"first pass downsampling failed\", \"err\", err)\n\treturn errors.Wrap(err, \"first pass of downsampling failed\")\n}","preventionTips":["Monitor free disk space on --data-dir (downsampling needs headroom for intermediates)","Keep --hash-func consistent with upload-time hash functions","Avoid killing the process mid-downsample; use graceful shutdown"],"tags":["thanos","downsampling","disk","corruption"],"backgroundTag":"file-write-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}