{"record":{"id":"f409ee8db8eff10e","repo":"Hmbown/CodeWhale","slug":"exact-library-metric-test-metric-test-name-test-n","errorCode":null,"errorMessage":"exact library metric test {metric_test_name(test_name)} ran zero tests","messagePattern":"exact library metric test (.+?) ran zero tests","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"scripts/measure-runtime-contract.py","lineNumber":52,"sourceCode":"        \"--ignored\",\n        \"--exact\",\n        \"--nocapture\",\n        \"--test-threads=1\",\n    ]\n\n\ndef run_metric(test_name: str, marker: str) -> dict:\n    cmd = metric_command(test_name)\n    proc = subprocess.run(cmd, text=True, capture_output=True, check=False)\n    sys.stderr.write(proc.stderr)\n    if proc.returncode != 0:\n        sys.stdout.write(proc.stdout)\n        proc.check_returncode()\n\n    combined = proc.stdout.splitlines() + proc.stderr.splitlines()\n    if re.search(r\"\\brunning\\s+0\\s+tests?\\b\", \"\\n\".join(combined)):\n        sys.stdout.write(proc.stdout)\n        raise RuntimeError(\n            f\"exact library metric test {metric_test_name(test_name)} ran zero tests\"\n        )\n    for line in combined:\n        if marker in line:\n            return json.loads(line.split(marker, 1)[1])\n\n    sys.stdout.write(proc.stdout)\n    raise RuntimeError(\n        f\"missing {marker.rstrip()} marker from exact library metric test \"\n        f\"{metric_test_name(test_name)}\"\n    )\n\n\ndef main() -> int:\n    tool_metrics = run_metric(\n        \"print_mode_tool_catalog_metrics\",\n        \"TOOL_CATALOG_METRICS \",\n    )","sourceCodeStart":34,"sourceCodeEnd":70,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/8880682c63083a91624de936797efa3ce9e498fd/scripts/measure-runtime-contract.py#L34-L70","documentation":"scripts/measure-runtime-contract.py builds the provider-free runtime-contract receipt by running one exact ignored library test (cargo test --locked -p codewhale-tui --lib core::engine::tests::<name> -- --ignored --exact --nocapture --test-threads=1) and parsing a marker line. This error means the test harness output contained 'running 0 tests', i.e. the exact filter selected nothing, so no metric could be collected. The script fails fast instead of emitting an empty receipt.","triggerScenarios":"The metric test was renamed, moved out of the core::engine::tests module, or deleted while scripts/measure-runtime-contract.py still names it; the test lost its #[ignore] attribute so the --ignored flag excludes it; the METRIC_TEST_MODULE constant or the test name passed to run_metric() (print_mode_tool_catalog_metrics, print_mode_runtime_contract_metrics, print_representative_runtime_context_metrics, print_skill_discovery_turn_metrics) has a typo or is stale.","commonSituations":"Refactoring or relocating tui tests (e.g. moving print_mode_tool_catalog_metrics to another module), removing #[ignore] during test cleanup, or adding a new run_metric() call in main() with a name that does not exactly match an existing #[ignore]d test.","solutions":["Run the exact command the script builds and confirm the test exists: cargo test --locked -p codewhale-tui --lib core::engine::tests::print_mode_tool_catalog_metrics -- --ignored --exact --nocapture","Restore #[ignore] on the metric test if it was removed -- the runner passes --ignored, so only ignored tests are selected","Update METRIC_TEST_MODULE or the test-name argument in scripts/measure-runtime-contract.py to the current module path and test name","If the test was intentionally removed, delete the corresponding run_metric() call in main()"],"exampleFix":"// crates/tui: before\n#[test]\nfn print_mode_tool_catalog_metrics() { ... }\n\n// after\n#[test]\n#[ignore]\nfn print_mode_tool_catalog_metrics() { ... }","handlingStrategy":"validation","validationCode":"# Before running the metric, confirm the exact ignored test exists:\nimport subprocess, sys\n\ndef metric_test_exists(test_name: str, module: str = \"core::engine::tests\") -> bool:\n    listing = subprocess.run(\n        [\"cargo\", \"test\", \"--locked\", \"-p\", \"codewhale-tui\", \"--lib\",\n         f\"{module}::{test_name}\", \"--\", \"--ignored\", \"--exact\", \"--list\"],\n        capture_output=True, text=True, check=False,\n    )\n    return f\"{module}::{test_name}: test\" in listing.stdout\n\nif not metric_test_exists(\"print_mode_tool_catalog_metrics\"):\n    sys.exit(\"metric test missing/renamed/not #[ignore]d -- fix before measuring\")","typeGuard":null,"tryCatchPattern":"try:\n    metrics = run_metric(\"print_mode_tool_catalog_metrics\", \"TOOL_CATALOG_METRICS \")\nexcept RuntimeError as e:\n    if \"ran zero tests\" in str(e):\n        # stale test name/module: re-check the test inventory, do not retry blindly\n        ...","preventionTips":["When renaming or moving a metric test, update METRIC_TEST_MODULE and the run_metric() names in scripts/measure-runtime-contract.py in the same change","Keep #[ignore] on every metric test -- the runner selects with --ignored","Run scripts/measure-runtime-contract.py in CI so name drift is caught at the commit that causes it"],"tags":["python","cargo","tests","metrics","ci"],"backgroundTag":null,"analyzedSha":"8880682c63083a91624de936797efa3ce9e498fd","analyzedAt":"2026-08-16T11:31:27.956Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}