{"record":{"id":"f409ee8db8eff10e","repo":"Hmbown/CodeWhale","slug":"exact-library-metric-test-metric-test-name-test-name-ran","errorCode":null,"errorMessage":"exact library metric test {metric_test_name(test_name)} ran zero tests","messagePattern":"exact library metric test (.+?) ran zero tests","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"scripts/measure-runtime-contract.py","lineNumber":52,"sourceCode":"        \"--ignored\",\n        \"--exact\",\n        \"--nocapture\",\n        \"--test-threads=1\",\n    ]\n\n\ndef run_metric(test_name: str, marker: str) -> dict:\n    cmd = metric_command(test_name)\n    proc = subprocess.run(cmd, text=True, capture_output=True, check=False)\n    sys.stderr.write(proc.stderr)\n    if proc.returncode != 0:\n        sys.stdout.write(proc.stdout)\n        proc.check_returncode()\n\n    combined = proc.stdout.splitlines() + proc.stderr.splitlines()\n    if re.search(r\"\\brunning\\s+0\\s+tests?\\b\", \"\\n\".join(combined)):\n        sys.stdout.write(proc.stdout)\n        raise RuntimeError(\n            f\"exact library metric test {metric_test_name(test_name)} ran zero tests\"\n        )\n    for line in combined:\n        if marker in line:\n            return json.loads(line.split(marker, 1)[1])\n\n    sys.stdout.write(proc.stdout)\n    raise RuntimeError(\n        f\"missing {marker.rstrip()} marker from exact library metric test \"\n        f\"{metric_test_name(test_name)}\"\n    )\n\n\ndef main() -> int:\n    tool_metrics = run_metric(\n        \"print_mode_tool_catalog_metrics\",\n        \"TOOL_CATALOG_METRICS \",\n    )","sourceCodeStart":34,"sourceCodeEnd":70,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/433685b2024e7bc4c99e1e2e326bcad39b4d9d65/scripts/measure-runtime-contract.py#L34-L70","documentation":"scripts/measure-runtime-contract.py runs an exact library metric test as a subprocess and parses a JSON marker from its output. This RuntimeError is raised when the combined stdout/stderr shows the test run reported 'running 0 tests', meaning the filter matched no tests and no metric was measured. The script refuses to emit a fabricated/empty metric, so it aborts loudly instead.","triggerScenarios":"The test name passed via metric_test_name/test_name no longer matches any test in the test binary (renamed or removed test), or the filter string passed to the test runner is misspelled or over-narrow, so cargo's test harness prints 'running 0 tests'.","commonSituations":"A test was renamed during refactoring but scripts/measure-runtime-contract.py still references the old name; a feature-gated test is compiled out so the filter matches nothing; running the script on a workspace subset where the crate containing the test is not built; typos when adding a new metric entry.","solutions":["Run the test filter manually (e.g. cargo test <filter>) and confirm at least one test matches; fix the test name in the script if it was renamed","Check that the crate containing the test is included in the workspace/packages being built and the required cargo features are enabled","Update the metric test list in scripts/measure-runtime-contract.py to reflect current test names","Re-run the script after cargo build to ensure the test binary is fresh"],"exampleFix":"// before (script references removed test)\nrun_metric(\"bench_exact_token_count\")\n// after\ncargo test exact_token_count  # verify it exists, then:\nrun_metric(\"exact_token_count\")","handlingStrategy":"validation","validationCode":"import subprocess\noutput = subprocess.run([\"cargo\", \"test\", test_filter, \"--\", \"--list\"], capture_output=True, text=True)\nif not output.stdout.strip():\n    raise SystemExit(f\"filter {test_filter!r} matches no tests; fix before running measure-runtime-contract.py\")","typeGuard":null,"tryCatchPattern":"try:\n    metric = run_metric(test_name)\nexcept RuntimeError as e:\n    if \"ran zero tests\" in str(e):\n        print(f\"test {test_name} not found; check rename/removal\")\n        sys.exit(2)\n    raise","preventionTips":["Keep the metric test list in the script in sync with test names (grep the test source when renaming)","Run cargo test <filter> manually before adding a new metric entry","Avoid marking metric tests #[ignore] or hiding them behind unenabled features"],"tags":["python","testing","subprocess","metrics"],"backgroundTag":"empty-result-set","analyzedSha":"433685b2024e7bc4c99e1e2e326bcad39b4d9d65","analyzedAt":"2026-09-15T12:24:24.634Z","contentChangedAt":"2026-09-15T12:24:24.634Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}