thanos-io/thanos · error

downsample block to window

Error message

downsample block %s to window %d

What it means

Wraps downsample.Downsample failures as errors.Wrapf(err, "downsample block %s to window %d", m.ULID, resolution). The actual aggregation of raw series into the coarser-resolution block failed — query/iterator errors, aggregation panics, or failure writing the resulting block.

Solutions

1. Inspect the inner error; if it is data corruption in the source block, delete/regenerate that block.
2. Give the downsampler more memory or reduce --downsample-concurrency to limit concurrent large aggregations.
3. Free disk space in --data-dir or point it at a larger volume.
4. Upgrade Thanos — several historical downsample bugs (chunk handling) were fixed in later releases.

if err := processDownsampling(...); err != nil {
    var rerr *compact.RetryError
    if errors.As(err, &rerr) { /* retry later */ }
    metrics.downsampleFailures.WithLabelValues(res).Inc()
    return errors.Wrapf(err, "downsample block %s to window %d", id, resolution)
}

When it happens

Trigger: downsample.Downsample(ctx, logger, m, b, dir, resolution) returns error: corrupt chunk data encountered while iterating, unsupported chunk encodings, out-of-memory while aggregating, or inability to create/write the result block in dir.

Common situations: Source blocks containing corrupted chunks written by older versions; OOM-killed downsampler on very large blocks; disk full in --data-dir when writing the downsampled output.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

• raw resolution must be higher than the minimum block size…
• 5m resolution retention must be higher than the minimum…
• sync before first pass of downsampling
• first pass of downsampling failed
• sync before second pass of downsampling

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/583e9422224cd339. Report an issue: GitHub.

Appendix: source

Thrown at cmd/thanos/downsample.go:387

	begin = time.Now()

	var pool chunkenc.Pool
	if m.Thanos.Downsample.Resolution == 0 {
		pool = chunkenc.NewPool()
	} else {
		pool = downsample.NewPool()
	}

	b, err := tsdb.OpenBlock(logutil.GoKitLogToSlog(logger), bdir, pool, nil)
	if err != nil {
		return errors.Wrapf(err, "open block %s", m.ULID)
	}
	defer runutil.CloseWithLogOnErr(log.With(logger, "outcome", "potential left mmap file handlers left"), b, "tsdb reader")

	id, err := downsample.Downsample(ctx, logger, m, b, dir, resolution)
	if err != nil {
		return errors.Wrapf(err, "downsample block %s to window %d", m.ULID, resolution)
	}
	resdir := filepath.Join(dir, id.String())

	downsampleDuration := time.Since(begin)
	level.Info(logger).Log("msg", "downsampled block",
		"from", m.ULID, "to", id, "duration", downsampleDuration, "duration_ms", downsampleDuration.Milliseconds())
	metrics.downsampleDuration.WithLabelValues(m.Thanos.ResolutionString()).Observe(downsampleDuration.Seconds())

	stats, err := block.GatherIndexHealthStats(ctx, logger, filepath.Join(resdir, block.IndexFilename), m.MinTime, m.MaxTime)
	if err == nil {
		err = stats.AnyErr()
	}
	if err != nil && !acceptMalformedIndex {
		return errors.Wrap(err, "output block index not valid")
	}

	meta, err := metadata.ReadFromDir(resdir)
	if err != nil {

View on GitHub (pinned to 35b8b99117)