{"record":{"id":"a83e400425c09bf2","repo":"thanos-io/thanos","slug":"sync-before-first-pass-of-downsampling","errorCode":null,"errorMessage":"sync before first pass of downsampling","messagePattern":"sync before first pass of downsampling","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cmd/thanos/compact.go","lineNumber":454,"sourceCode":"\n\t\tcompact.BestEffortCleanAbortedPartialUploads(ctx, logger, sy.Partial(), insBkt, compactMetrics.partialUploadDeleteAttempts, compactMetrics.blocksCleaned, compactMetrics.blockCleanupFailures, ignoreDeletionMarkFilter.DeletionMarkBlocks())\n\t\tcompactMetrics.cleanups.Inc()\n\n\t\treturn nil\n\t}\n\n\tcompactMainFn := func() error {\n\t\tif err := compactor.Compact(ctx); err != nil {\n\t\t\treturn errors.Wrap(err, \"compaction\")\n\t\t}\n\n\t\tif !conf.disableDownsampling {\n\t\t\t// After all compactions are done, work down the downsampling backlog.\n\t\t\t// We run two passes of this to ensure that the 1h downsampling is generated\n\t\t\t// for 5m downsamplings created in the first run.\n\t\t\tlevel.Info(logger).Log(\"msg\", \"start first pass of downsampling\")\n\t\t\tif err := sy.SyncMetas(ctx); err != nil {\n\t\t\t\treturn errors.Wrap(err, \"sync before first pass of downsampling\")\n\t\t\t}\n\n\t\t\tfilteredMetas := sy.Metas()\n\t\t\tnoDownsampleBlocks := noDownsampleMarkerFilter.NoDownsampleMarkedBlocks()\n\t\t\tfor ul := range noDownsampleBlocks {\n\t\t\t\tdelete(filteredMetas, ul)\n\t\t\t}\n\n\t\t\tfor _, meta := range filteredMetas {\n\t\t\t\tresolutionLabel := meta.Thanos.ResolutionString()\n\t\t\t\tdownsampleMetrics.downsamples.WithLabelValues(resolutionLabel)\n\t\t\t\tdownsampleMetrics.downsampleFailures.WithLabelValues(resolutionLabel)\n\t\t\t}\n\n\t\t\tif err := downsampleBucket(\n\t\t\t\tctx,\n\t\t\t\tlogger,\n\t\t\t\tdownsampleMetrics,","sourceCodeStart":436,"sourceCodeEnd":472,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/cmd/thanos/compact.go#L436-L472","documentation":"Wrapper error in runCompact's downsampling first pass (cmd/thanos/compact.go:454) around sy.SyncMetas(ctx). The bucket metadata syncer failed to refresh block metas from the object store before downsampling began, so the process cannot build a reliable filtered block list.","triggerScenarios":"sy.SyncMetas returning an error: object store list/get failures, network interruption, throttling (429), or malformed meta.json objects during sync.","commonSituations":"S3/GCS outages or rate limiting during large buckets, expired credentials, proxy/firewall interruptions between Thanos and the store.","solutions":["Check the wrapped inner error for the failing object store operation","Retry thanos compact; SyncMetas is idempotent and retried on the next run","Verify object store credentials and network path","Enable request logging/metrics on the bucket client to identify throttling","Increase request timeouts/retries on the object store client config"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"if err := sy.SyncMetas(ctx); err != nil {\n\tif errors.Is(err, context.Canceled) { return err }\n\treturn errors.Wrap(err, \"sync before first pass of downsampling\")\n}","preventionTips":["Configure bucket client retries/backoff for 5xx and 429 responses","Use valid, long-lived credentials for long compaction cycles","Alert on object-store error rate metrics during compaction windows"],"tags":["thanos","object-storage","sync","downsampling"],"backgroundTag":"http-request-failed","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}