thanos-io/thanos · error

writing series from to

Error message

writing series from %v to %v

What it means

Wraps compactv2 compactor WriteSeries failing while rewriting series from the source block into a new block. WriteSeries iterates the source index and writes filtered/relabeled series to a destination; failures come from relabel config application errors, malformed index data, context cancellation, or disk write errors during compaction.

Solutions

1. Validate the relabel YAML/regex with promtool before passing --rewrite.
2. Increase tmpDir disk space (point --tmp.dir to a large volume) and retry.
3. Check the wrapped cause: if it is context canceled, re-run without interruption; if index corruption, restore/verify the block first.

Example fix

// before (invalid relabel regex)
--rewrite="- action: replace\n  regex: '(.*[ )'\n  ..."
// after (validated)
--rewrite="- action: replace\n  regex: '(.*)'\n  replacement: '$1'\n  ..."

if err := relabel.Validate(labels.FromStrings("__name__", "test"), relabelCfgs...); err != nil {
    return fmt.Errorf("invalid relabel config: %w", err)
}

if err := comp.WriteSeries(ctx, readers, d, p, modifiers...); err != nil {
    if errors.Is(err, context.Canceled) { return err }
    return errors.Wrapf(err, "writing series from %v to %v", id, newID)
}

When it happens

Trigger: Invalid relabel config (bad regex, duplicate labels after replacement) passed via --rewrite; the source block index contains corrupt or out-of-order series; tmpDir runs out of space while writing; context canceled (Ctrl-C, timeout).

Common situations: Relabel regex with catastrophic backtracking or invalid syntax; rewriting very large blocks with insufficient tmpDir disk; series selectors in --delete that cause edge-case iteration issues; operator aborting the long-running rewrite.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• create bucket compactor
• compaction
• critical error detected
• error executing compaction
• could not group metadata for compaction

AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07). Data as JSON: /api/errors/c83c120a655c6232. Report an issue: GitHub.

Appendix: source

Thrown at cmd/thanos/tools_bucket.go:1302

					changeLog = compactv2.NewChangeLog(f)
					level.Info(logger).Log("msg", "changelog will be available", "file", filepath.Join(tbc.tmpDir, newID.String(), "change.log"))
				}

				d, err := block.NewDiskWriter(ctx, logger, filepath.Join(tbc.tmpDir, newID.String()))
				if err != nil {
					return err
				}

				var comp *compactv2.Compactor
				if tbc.dryRun {
					comp = compactv2.NewDryRun(tbc.tmpDir, logger, changeLog, chunkPool)
				} else {
					comp = compactv2.New(tbc.tmpDir, logger, changeLog, chunkPool)
				}

				level.Info(logger).Log("msg", "starting rewrite for block", "source", id, "new", newID, "toDelete", string(deletionsYaml), "toRelabel", string(relabelYaml))
				if err := comp.WriteSeries(ctx, []block.Reader{b}, d, p, modifiers...); err != nil {
					return errors.Wrapf(err, "writing series from %v to %v", id, newID)
				}

				if tbc.dryRun {
					level.Info(logger).Log("msg", "dry run finished. Changes should be printed to stderr", "Block ID", id)
					continue
				}

				level.Info(logger).Log("msg", "wrote new block after modifications; flushing", "source", id, "new", newID)
				meta.Stats, err = d.Flush()
				if err != nil {
					return errors.Wrap(err, "flush")
				}
				if err := meta.WriteToDir(logger, filepath.Join(tbc.tmpDir, newID.String())); err != nil {
					return err
				}

				level.Info(logger).Log("msg", "uploading new block", "source", id, "new", newID)
				if tbc.promBlocks {

View on GitHub (pinned to 35b8b99117)