tursodatabase/turso · error · ValueError

sweep files must match benchmark, sample budget, fixture…

Error message

sweep files must match benchmark, sample budget, fixture and engine/mode series

What it means

After reading each sweep file, read_sweep compares its remaining configuration tuple and sample series against the first file's. It raises this ValueError when files disagree on benchmark configuration (benchmark, sample budget, fixture) or on the set of engine/mode series, since a combined connection-sweep plot would be meaningless across different setups.

Solutions

  1. Regenerate all sweep files with the same benchmark, sample budget, fixture, and engine/mode set.
  2. Remove the sweep files that came from a different configuration.
  3. Align the series by rerunning the missing engine/mode combinations so every file has identical series.

Example fix

# before
python plot-fts.py sweep-old-fixture/c8.csv sweep-new-fixture/c16.csv
# after
python plot-fts.py sweep-v2/c8.csv sweep-v2/c16.csv
Defensive patterns

Strategy: validation

Validate before calling

import csv
def config_of(p):
    r = csv.DictReader(open(p))
    return tuple(next(r)[k] for k in CONFIGURATION)
assert len({config_of(p) for p in paths}) == 1, 'all sweep files must share one configuration'

Try / catch

try:
    config, points = read_sweep(paths)
except ValueError as e:
    if 'must match benchmark' in str(e):
        print('Fix: regenerate all sweep files with the same benchmark/fixture/series')
    else:
        raise

Prevention

When it happens

Trigger: Calling read_sweep with CSVs generated by different plot-fts configurations (different benchmark, percentile/sample budget, fixture) or containing different engine/mode columns than the first file.

Common situations: Mixing result files from before and after a plot-fts/benchmark change; plotting sweep files generated against different database fixtures; renaming engines in one run but not others.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of tursodatabase/turso@8d4a589f8d (2026-09-20). Data as JSON: /api/errors/bf4e184c3e582f12. Report an issue: GitHub.

Appendix: source

Thrown at perf/fts/plot/plot-fts.py:61

        if args.sweep:
            plot_sweep(configuration, samples, output, args.relative)
        else:
            plot(configuration, samples, output)
        print(f"wrote {output}")


def read_sweep(paths, percentile="p50"):
    configuration = None
    points = {}
    for path in paths:
        current, samples = read_runs([path], percentile)
        connections = int(current.pop("connections"))
        if connections <= 0 or connections in points:
            raise ValueError("each sweep file must have a distinct positive connection count")
        if configuration is None:
            configuration = current
        if current != configuration or (points and set(samples) != set(next(iter(points.values())))):
            raise ValueError("sweep files must match benchmark, sample budget, fixture and engine/mode series")
        points[connections] = samples
    if not points:
        raise ValueError("no sweep results found")
    return configuration, dict(sorted(points.items()))


def read_runs(paths, percentile="p50"):
    configuration = None
    samples = {}
    expected_series = None
    for path in paths:
        current, runs = read_file(path, percentile, samples)
        if configuration is None:
            configuration = current
        if configuration != current:
            raise ValueError("plot only one configuration at a time")
        for run in runs.values():
            if expected_series is None:

View on GitHub (pinned to 8d4a589f8d)