{"record":{"id":"f5d35aac86ec0a92","repo":"thanos-io/thanos","slug":"compaction","errorCode":null,"errorMessage":"compaction","messagePattern":"compaction","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cmd/thanos/compact.go","lineNumber":445,"sourceCode":"\tif retentionByResolution[compact.ResolutionLevel1h].Milliseconds() != 0 {\n\t\tlevel.Info(logger).Log(\"msg\", \"retention policy of 1 hour aggregated samples is enabled\", \"duration\", retentionByResolution[compact.ResolutionLevel1h])\n\t}\n\n\tvar cleanMtx sync.Mutex\n\t// TODO(GiedriusS): we could also apply retention policies here but the logic would be a bit more complex.\n\tcleanPartialMarked := func() error {\n\t\tcleanMtx.Lock()\n\t\tdefer cleanMtx.Unlock()\n\n\t\tcompact.BestEffortCleanAbortedPartialUploads(ctx, logger, sy.Partial(), insBkt, compactMetrics.partialUploadDeleteAttempts, compactMetrics.blocksCleaned, compactMetrics.blockCleanupFailures, ignoreDeletionMarkFilter.DeletionMarkBlocks())\n\t\tcompactMetrics.cleanups.Inc()\n\n\t\treturn nil\n\t}\n\n\tcompactMainFn := func() error {\n\t\tif err := compactor.Compact(ctx); err != nil {\n\t\t\treturn errors.Wrap(err, \"compaction\")\n\t\t}\n\n\t\tif !conf.disableDownsampling {\n\t\t\t// After all compactions are done, work down the downsampling backlog.\n\t\t\t// We run two passes of this to ensure that the 1h downsampling is generated\n\t\t\t// for 5m downsamplings created in the first run.\n\t\t\tlevel.Info(logger).Log(\"msg\", \"start first pass of downsampling\")\n\t\t\tif err := sy.SyncMetas(ctx); err != nil {\n\t\t\t\treturn errors.Wrap(err, \"sync before first pass of downsampling\")\n\t\t\t}\n\n\t\t\tfilteredMetas := sy.Metas()\n\t\t\tnoDownsampleBlocks := noDownsampleMarkerFilter.NoDownsampleMarkedBlocks()\n\t\t\tfor ul := range noDownsampleBlocks {\n\t\t\t\tdelete(filteredMetas, ul)\n\t\t\t}\n\n\t\t\tfor _, meta := range filteredMetas {","sourceCodeStart":427,"sourceCodeEnd":463,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/cmd/thanos/compact.go#L427-L463","documentation":"Wrapper error from the compactMainFn closure in runCompact (cmd/thanos/compact.go:445) around compactor.Compact(ctx). It means a compaction iteration over bucket blocks failed; the real cause is nested inside (e.g. network errors to object storage, corrupt block index, meta.json issues).","triggerScenarios":"Any error returned by compactor.Compact(ctx): object store unavailability, partial/corrupt blocks, out-of-order block conflicts, grouping/plan errors.","commonSituations":"S3/GCS transient failures or rate limits, blocks with checksum mismatches, blocks uploaded out of order (skipped or erroring depending on flags), permission errors on bucket.","solutions":["Read the wrapped inner error to find the failing block or bucket operation","Retry the command — compaction is resumable; transient object-store errors often resolve","Inspect the problematic block in the bucket (meta.json, index checksums) and remove/repair corrupt blocks if confirmed","Verify object store credentials/permissions and network connectivity","Consider --skip-block-with-out-of-order-chunks or fixing uploaders producing out-of-order chunks"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// ensure bucket reachable before starting\nif _, err := insBkt.Iter(ctx, \"\", func(string) error { return nil }); err != nil {\n\treturn errors.Wrap(err, \"bucket not reachable before compaction\")\n}","typeGuard":null,"tryCatchPattern":"if err := compactor.Compact(ctx); err != nil {\n\tif errors.Is(err, context.Canceled) {\n\t\treturn err // shutdown, do not retry\n\t}\n\tlevel.Error(logger).Log(\"msg\", \"compaction failed\", \"err\", err)\n\treturn errors.Wrap(err, \"compaction\") // outer loop/Halt will retry\n}","preventionTips":["Run compactor as a single instance with retries on transient store errors","Monitor block corruption metrics and alert early","Use --debug.halt-on-error only when you want fail-fast semantics","Ensure uploaders never produce out-of-order chunks"],"tags":["thanos","compaction","object-storage"],"backgroundTag":"http-request-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}