{"record":{"id":"3d087e8926e8c5a6","repo":"t8y2/dbx","slug":"bench-order-must-contain-exactly-join-comman","errorCode":null,"errorMessage":"BENCH_ORDER must contain exactly: {','.join(commands)}","messagePattern":"BENCH_ORDER must contain exactly: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"agents/drivers/iotdb/bench/run.py","lineNumber":227,"sourceCode":"        \"rounds\": rounds,\n    }\n\n\ndef percentile(values: list[float], fraction: float) -> float:\n    index = max(0, min(len(values) - 1, int(len(values) * fraction + 0.999999) - 1))\n    return values[index]\n\n\ndef ensure_artifacts() -> None:\n    for path in (JDBC_JAR, GO_BINARY):\n        if not path.is_file():\n            raise FileNotFoundError(path)\n\n\ndef benchmark_order(commands: dict[str, list[str]]) -> list[str]:\n    names = [name.strip() for name in os.getenv(\"BENCH_ORDER\", \"jdbc,go\").split(\",\") if name.strip()]\n    if len(names) != len(commands) or set(names) != set(commands):\n        raise ValueError(f\"BENCH_ORDER must contain exactly: {','.join(commands)}\")\n    return names\n\n\ndef env_int(name: str, fallback: int) -> int:\n    value = int(os.getenv(name, str(fallback)))\n    if value <= 0:\n        raise ValueError(f\"{name} must be positive\")\n    return value\n\n\ndef env_float(name: str, fallback: float) -> float:\n    value = float(os.getenv(name, str(fallback)))\n    if value <= 0:\n        raise ValueError(f\"{name} must be positive\")\n    return value\n\n\ndef env_bool(name: str, fallback: bool) -> bool:","sourceCodeStart":209,"sourceCodeEnd":245,"githubUrl":"https://github.com/t8y2/dbx/blob/c0390bff16418b651f4728520d99adf8ce48829a/agents/drivers/iotdb/bench/run.py#L209-L245","documentation":"benchmark_order parses the BENCH_ORDER environment variable, which must list exactly the same benchmark names as the available commands (by default jdbc,go). If the comma-separated list doesn't match the command set exactly, it raises ValueError naming the required set. This exists to let users reorder benchmarks without silently dropping or duplicating one.","triggerScenarios":"Setting BENCH_ORDER to something other than a permutation of the command keys — e.g. BENCH_ORDER=jdbc (missing go), BENCH_ORDER=\"jdbc, go, extra\" (extra name), duplicates, or empty entries leading to count mismatch.","commonSituations":"Typo in a benchmark name; copying BENCH_ORDER from an older script version when a new benchmark was added; trailing/extra commas producing empty names; whitespace handled but misspelled names not.","solutions":["Set BENCH_ORDER to exactly the required names in the desired order, e.g. BENCH_ORDER=go,jdbc.","Unset BENCH_ORDER to use the default 'jdbc,go'.","Check the valid keys by inspecting the commands dict / error message listing the expected set.","Update stale CI configs or docs to include any newly added benchmark names."],"exampleFix":"# before\nBENCH_ORDER=jdbc python run.py\n# after\nBENCH_ORDER=jdbc,go python run.py","handlingStrategy":"validation","validationCode":"import os\nnames = [n.strip() for n in os.getenv(\"BENCH_ORDER\", \"jdbc,go\").split(\",\") if n.strip()]\nassert set(names) == {\"jdbc\", \"go\"}, f\"BENCH_ORDER must be a permutation of jdbc,go, got {names}\"","typeGuard":null,"tryCatchPattern":"try:\n    order = benchmark_order(commands)\nexcept ValueError as e:\n    print(e); order = list(commands)  # fall back to default order","preventionTips":["Only set BENCH_ORDER to a comma-separated permutation of the printed valid names","Unset the variable to use defaults when unsure","Keep benchmark lists in CI configs in sync with the script","Avoid duplicate or empty entries in the list"],"tags":["python","env-var","validation","benchmark"],"backgroundTag":"invalid-env-var-value","analyzedSha":"c0390bff16418b651f4728520d99adf8ce48829a","analyzedAt":"2026-09-05T23:05:10.900Z","contentChangedAt":"2026-09-05T23:05:10.900Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}