{"record":{"id":"23fa99d2136027c2","repo":"rohitg00/ai-engineering-from-scratch","slug":"mode-decision-needs-rationale-repetition-change","errorCode":null,"errorMessage":"mode decision needs rationale, repetition, change policy, and rejections","messagePattern":"mode decision needs rationale, repetition, change policy, and rejections","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"certifications/claude/lessons/02-model-selection-and-token-economics/code/main.py","lineNumber":213,"sourceCode":"        if selected != best[\"id\"]:\n            errors.append(\"selectedConfiguration must be the least costly passing mode\")\n\n    decision = experiment.get(\"decision\")\n    if (\n        not isinstance(decision, dict)\n        or not str(decision.get(\"why\", \"\")).strip()\n        or not str(decision.get(\"repeatPolicy\", \"\")).strip()\n        or not str(decision.get(\"changePolicy\", \"\")).strip()\n        or not _nonempty_strings(decision.get(\"rejected\"))\n    ):\n        errors.append(\"mode decision needs rationale, repetition, change policy, and rejections\")\n    return errors\n\n\ndef summarize_mode_trials(experiment: dict[str, Any]) -> dict[str, Any]:\n    errors = validate_mode_trials(experiment)\n    if errors:\n        raise ValueError(\"; \".join(errors))\n    selected = next(\n        item for item in experiment[\"configurations\"] if item[\"id\"] == experiment[\"selectedConfiguration\"]\n    )\n    return {\n        \"selectedConfiguration\": selected[\"id\"],\n        \"settings\": selected[\"settings\"],\n        \"minimumQuality\": selected[\"summary\"][\"minimumQuality\"],\n        \"p95LatencyMs\": selected[\"summary\"][\"p95LatencyMs\"],\n        \"meanCostUnits\": selected[\"summary\"][\"meanCostUnits\"],\n        \"supportedConfigurations\": sum(\n            item[\"support\"][\"status\"] == \"docs-supported\" for item in experiment[\"configurations\"]\n        ),\n    }\n\n\ndef _validate_repeated_runs(configuration_id: str, runs: Any, minimum_run_count: int) -> list[str]:\n    if not isinstance(runs, list) or len(runs) < minimum_run_count:\n        return [f\"{configuration_id} requires at least {minimum_run_count} repeated runs\"]","sourceCodeStart":195,"sourceCodeEnd":231,"githubUrl":"https://github.com/rohitg00/ai-engineering-from-scratch/blob/39ea8a1c6d0b61f071226eff7ede4d4105fed820/certifications/claude/lessons/02-model-selection-and-token-economics/code/main.py#L195-L231","documentation":"Completeness check on a mode-trials decision block: experiment.decision must carry a non-empty 'why', 'repeatPolicy', 'changePolicy', and a non-empty list of 'rejected' alternatives. One of these is missing or blank, so the experiment's conclusion is not auditable and validation fails.","triggerScenarios":"Thrown at certifications/claude/lessons/02-model-selection-and-token-economics/code/main.py:213 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Fill in why/repeatPolicy/changePolicy and list the rejected modes with reasons","Document the repetition count and what would change the chosen mode","Treat the decision block as mandatory in the schema and validate it before summarizing"],"exampleFix":null,"handlingStrategy":"validation","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"39ea8a1c6d0b61f071226eff7ede4d4105fed820","analyzedAt":"2026-08-26T03:13:46.626Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}