{"record":{"id":"b57560eea4db8d28","repo":"tursodatabase/turso","slug":"unknown-benchmark","errorCode":null,"errorMessage":"unknown benchmark","messagePattern":"unknown benchmark","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"perf/fts/plot/plot-fts.py","lineNumber":140,"sourceCode":"    queries = int(row[\"queries\"])\n    positive_number(queries)\n    if row[\"benchmark\"] != \"memory\":\n        seconds = positive_number(row[\"seconds\"])\n    if row[\"benchmark\"] == \"memory\":\n        if row[\"engine\"] != \"turso\" or queries != int(row[\"requested_queries\"]) * int(row[\"connections\"]):\n            raise ValueError(\"heap results require Turso and the requested queries per connection\")\n        value = float(row[\"peak_heap_bytes\"]) / (1024 * 1024)\n    elif row[\"benchmark\"] == \"search\":\n        if not 0 < int(row[\"connections\"]) <= queries or queries != int(row[\"requested_queries\"]):\n            raise ValueError(\"search results require at least one sample per connection and the requested query count\")\n        value = float(row[f\"{percentile}_ms\"])\n    elif row[\"benchmark\"] == \"throughput\":\n        minimum = positive_number(row[\"min_seconds\"])\n        if seconds < minimum:\n            raise ValueError(\"throughput results must meet a positive minimum duration\")\n        value = queries / seconds\n    else:\n        raise ValueError(\"unknown benchmark\")\n    return positive_number(value)\n\n\ndef positive_number(value):\n    value = float(value)\n    if not np.isfinite(value) or value <= 0:\n        raise ValueError(\"measurements must be finite and positive\")\n    return value\n\n\ndef plot_sweep(configuration, points, output, relative=False):\n    import matplotlib\n\n    matplotlib.use(\"Agg\")\n    import matplotlib.pyplot as plt\n    from matplotlib.ticker import FuncFormatter, MaxNLocator, NullLocator\n\n    importlib.import_module(\"scienceplots\")","sourceCodeStart":122,"sourceCodeEnd":158,"githubUrl":"https://github.com/tursodatabase/turso/blob/8d4a589f8d13ac184700d2a8f724f27e1995be3b/perf/fts/plot/plot-fts.py#L122-L158","documentation":"read_measurement encountered a value in the benchmark column it does not recognize, so it cannot interpret the row's measurement (seconds vs peak_heap_bytes vs search-count rules). This is a generic validation guard: the CSV's benchmark column holds a value other than the supported 'memory' and 'search' kinds, usually a typo or a new benchmark kind the plotter was not updated for.","triggerScenarios":"CSV contains benchmark='latency' or a typo like 'thruput'; a new benchmark kind added by the harness but not the plot script.","commonSituations":"Renamed benchmark in the collector script; hand-edited CSV; plotting an unrelated benchmark's output with this tool.","solutions":["Fix the benchmark value in the CSV to one of memory|search|throughput","Add handling for the new benchmark kind in read_measurement()","Use the correct plotting tool for that benchmark"],"exampleFix":"// before (CSV)\nbenchmark=latency,...\n// after\nbenchmark=search,...","handlingStrategy":"validation","validationCode":"KNOWN_BENCHMARKS = {\"memory\", \"search\", \"throughput\"}\ndef validate_benchmark(row):\n    if row[\"benchmark\"] not in KNOWN_BENCHMARKS:\n        raise ValueError(f\"unknown benchmark {row['benchmark']}\")","typeGuard":"def is_known_benchmark(row) -> bool:\n    return row.get(\"benchmark\") in {\"memory\", \"search\", \"throughput\"}","tryCatchPattern":"try:\n    configuration, runs = read_runs(path, percentile)\nexcept ValueError as e:\n    print(f\"unknown benchmark kind: {e}\")","preventionTips":["Use only memory|search|throughput labels in results files","Add new benchmark handling to plot-fts.py when extending the harness","Keep the collector and plotter benchmark vocabularies in sync"],"tags":["python","benchmark","unsupported-value"],"backgroundTag":"invalid-enum-value","analyzedSha":"8d4a589f8d13ac184700d2a8f724f27e1995be3b","analyzedAt":"2026-09-20T13:18:14.658Z","contentChangedAt":"2026-09-20T13:18:14.658Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}