{"record":{"id":"d6e063e5a04600b8","repo":"Hmbown/CodeWhale","slug":"missing-marker-rstrip-marker-from-exact-library-metric-test","errorCode":null,"errorMessage":"missing {marker.rstrip()} marker from exact library metric test {metric_test_name(test_name)}","messagePattern":"missing (.+?) marker from exact library metric test (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"scripts/measure-runtime-contract.py","lineNumber":60,"sourceCode":"    cmd = metric_command(test_name)\n    proc = subprocess.run(cmd, text=True, capture_output=True, check=False)\n    sys.stderr.write(proc.stderr)\n    if proc.returncode != 0:\n        sys.stdout.write(proc.stdout)\n        proc.check_returncode()\n\n    combined = proc.stdout.splitlines() + proc.stderr.splitlines()\n    if re.search(r\"\\brunning\\s+0\\s+tests?\\b\", \"\\n\".join(combined)):\n        sys.stdout.write(proc.stdout)\n        raise RuntimeError(\n            f\"exact library metric test {metric_test_name(test_name)} ran zero tests\"\n        )\n    for line in combined:\n        if marker in line:\n            return json.loads(line.split(marker, 1)[1])\n\n    sys.stdout.write(proc.stdout)\n    raise RuntimeError(\n        f\"missing {marker.rstrip()} marker from exact library metric test \"\n        f\"{metric_test_name(test_name)}\"\n    )\n\n\ndef main() -> int:\n    tool_metrics = run_metric(\n        \"print_mode_tool_catalog_metrics\",\n        \"TOOL_CATALOG_METRICS \",\n    )\n    prompt_metrics = run_metric(\n        \"print_mode_runtime_contract_metrics\",\n        \"RUNTIME_CONTRACT_METRICS \",\n    )\n    representative_context_metrics = run_metric(\n        \"print_representative_runtime_context_metrics\",\n        \"REPRESENTATIVE_CONTEXT_METRICS \",\n    )","sourceCodeStart":42,"sourceCodeEnd":78,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/433685b2024e7bc4c99e1e2e326bcad39b4d9d65/scripts/measure-runtime-contract.py#L42-L78","documentation":"scripts/measure-runtime-contract.py runs a metric test subprocess and expects one output line containing a sentinel marker followed by JSON metrics. This RuntimeError is raised when the process succeeded and ran tests, but no line contained the expected marker, so no metric payload could be parsed.","triggerScenarios":"The test ran (not zero tests) but did not print the marker line — e.g. the test's metric-printing code was removed or renamed, the test was skipped/ignored so it never executed its print, or stdout/stderr buffering or encoding mangled the marker line.","commonSituations":"Refactoring a benchmark/metric test and forgetting to keep the println! with the agreed marker prefix; marking the test #[ignore]; marker string divergence between the script and the test after an edit; the test panics partway (though that usually surfaces as a nonzero exit first).","solutions":["Confirm the test actually prints a line containing the marker (grep the test source for the marker prefix)","Re-run the test manually and inspect its raw output for the marker line","If the marker format changed, update either the script's marker or the test's print statement so they match again","Check the test is not ignored or feature-gated out at runtime"],"exampleFix":"// before (test no longer prints marker)\nprintln!(\"RESULT {}\", json)\n// after (match the script's marker)\nprintln!(\"{}{}\", MARKER, json)","handlingStrategy":"validation","validationCode":"import subprocess\nout = subprocess.run([\"cargo\", \"test\", test_name, \"--\", \"--nocapture\"], capture_output=True, text=True)\nassert marker in out.stdout + out.stderr, f\"test {test_name} does not emit marker {marker!r}\"","typeGuard":null,"tryCatchPattern":"try:\n    metric = run_metric(test_name)\nexcept RuntimeError as e:\n    if \"marker\" in str(e):\n        print(f\"{test_name} ran but never printed {marker!r}; restore the metric print\")\n        sys.exit(3)\n    raise","preventionTips":["Keep the marker println in a shared helper/macro so refactors can't silently drop it","Never #[ignore] metric tests; gate whole metric entries instead","When changing the marker format, update script and test in the same commit"],"tags":["python","testing","subprocess","parsing"],"backgroundTag":"unexpected-response-shape","analyzedSha":"433685b2024e7bc4c99e1e2e326bcad39b4d9d65","analyzedAt":"2026-09-15T12:24:24.634Z","contentChangedAt":"2026-09-15T12:24:24.634Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}