{"record":{"id":"3e965a32a8ffc0f9","repo":"Hmbown/CodeWhale","slug":"invalid-fail-step-value","errorCode":null,"errorMessage":"invalid --fail-step '{value}'","messagePattern":"invalid --fail-step '(.+?)'","errorType":"validation","errorClass":"anyhow::Error","httpStatus":null,"severity":"warning","filePath":"crates/tui/src/lib.rs","lineNumber":2723,"sourceCode":"        );\n    }\n    std::fs::File::open(path)\n        .map_err(|error| anyhow!(\"could not securely open {}: {error}\", path.display()))\n}\n\n/// Generate shell completions for the given shell\nfn generate_completions(shell: Shell) {\n    let mut cmd = Cli::command();\n    let name = cmd.get_name().to_string();\n    generate(shell, &mut cmd, name, &mut io::stdout());\n}\n\n/// Run the offline evaluation harness (no network/LLM calls).\nfn run_eval(args: EvalArgs) -> Result<()> {\n    let fail_step = match args.fail_step.as_deref() {\n        Some(value) => ScenarioStepKind::parse(value)\n            .map(Some)\n            .ok_or_else(|| anyhow!(\"invalid --fail-step '{value}'\"))?,\n        None => None,\n    };\n\n    let config = EvalHarnessConfig {\n        fail_step,\n        shell_command: args.shell_command,\n        shell_expect_token: args.shell_expect_token,\n        max_output_chars: args.max_output_chars,\n        record_dir: args.record.clone(),\n        ..EvalHarnessConfig::default()\n    };\n\n    let harness = EvalHarness::new(config);\n    let run = harness.run().context(\"evaluation harness failed\")?;\n    let report = run.to_report();\n\n    if args.json {\n        let json = serde_json::to_string_pretty(&report)?;","sourceCodeStart":2705,"sourceCodeEnd":2741,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/8880682c63083a91624de936797efa3ce9e498fd/crates/tui/src/lib.rs#L2705-L2741","documentation":"codewhale eval --fail-step <value> injects a deliberate failure into the offline evaluation harness. The value must parse via ScenarioStepKind::parse, which accepts exactly: list, read, search, grep, edit, patch, apply_patch, bash, shell, exec (case-insensitive, surrounding whitespace trimmed). Anything else fails with \"invalid --fail-step '{value}'\". The harness models six step kinds: List, Read, Search, Edit, ApplyPatch, Bash.","triggerScenarios":"Passing a token outside the accepted set — for example --fail-step write, --fail-step file (the agent tool name, not a step kind), or --fail-step apply-patch (hyphen instead of underscore).","commonSituations":"Assuming the flag takes any tool name; hyphenation differences (apply-patch vs apply_patch); misspellings. Case and trailing spaces are tolerated.","solutions":["Use one of the accepted tokens: list, read, search, grep, edit, patch, apply_patch, bash, shell, exec","For the apply-patch step the token is apply_patch or patch — underscore, no hyphen","Run codewhale eval --help to re-check the flag after upgrading","If a new step kind is genuinely needed, extend ScenarioStepKind in crates/tui/src/eval.rs"],"exampleFix":"# before\ncodewhale eval --fail-step apply-patch\n\n# after\ncodewhale eval --fail-step apply_patch","handlingStrategy":"validation","validationCode":"const VALID_FAIL_STEPS: &[&str] = &[\n    \"list\", \"read\", \"search\", \"grep\", \"edit\",\n    \"patch\", \"apply_patch\", \"bash\", \"shell\", \"exec\",\n];\nif let Some(value) = &args.fail_step {\n    let ok = VALID_FAIL_STEPS.contains(&value.trim().to_lowercase().as_str());\n    assert!(ok, \"invalid --fail-step '{value}'\");\n}","typeGuard":"fn is_valid_fail_step(value: &str) -> bool {\n    matches!(\n        value.trim().to_lowercase().as_str(),\n        \"list\" | \"read\" | \"search\" | \"grep\" | \"edit\"\n            | \"patch\" | \"apply_patch\" | \"bash\" | \"shell\" | \"exec\"\n    )\n}","tryCatchPattern":null,"preventionTips":["Use underscore forms (apply_patch), never hyphens","Pass harness step names, not agent tool names","Re-check the accepted list in ScenarioStepKind::parse after upgrades"],"tags":["rust","cli","eval","argument-validation"],"backgroundTag":null,"analyzedSha":"8880682c63083a91624de936797efa3ce9e498fd","analyzedAt":"2026-08-16T11:31:27.956Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}