Hmbown/CodeWhale · warning · anyhow::Error

invalid --fail-step '{value}'

Error message

invalid --fail-step '{value}'

What it means

codewhale eval --fail-step <value> injects a deliberate failure into the offline evaluation harness. The value must parse via ScenarioStepKind::parse, which accepts exactly: list, read, search, grep, edit, patch, apply_patch, bash, shell, exec (case-insensitive, surrounding whitespace trimmed). Anything else fails with "invalid --fail-step '{value}'". The harness models six step kinds: List, Read, Search, Edit, ApplyPatch, Bash.

Source

Thrown at crates/tui/src/lib.rs:2723

        );
    }
    std::fs::File::open(path)
        .map_err(|error| anyhow!("could not securely open {}: {error}", path.display()))
}

/// Generate shell completions for the given shell
fn generate_completions(shell: Shell) {
    let mut cmd = Cli::command();
    let name = cmd.get_name().to_string();
    generate(shell, &mut cmd, name, &mut io::stdout());
}

/// Run the offline evaluation harness (no network/LLM calls).
fn run_eval(args: EvalArgs) -> Result<()> {
    let fail_step = match args.fail_step.as_deref() {
        Some(value) => ScenarioStepKind::parse(value)
            .map(Some)
            .ok_or_else(|| anyhow!("invalid --fail-step '{value}'"))?,
        None => None,
    };

    let config = EvalHarnessConfig {
        fail_step,
        shell_command: args.shell_command,
        shell_expect_token: args.shell_expect_token,
        max_output_chars: args.max_output_chars,
        record_dir: args.record.clone(),
        ..EvalHarnessConfig::default()
    };

    let harness = EvalHarness::new(config);
    let run = harness.run().context("evaluation harness failed")?;
    let report = run.to_report();

    if args.json {
        let json = serde_json::to_string_pretty(&report)?;

View on GitHub (pinned to 8880682c63)

Solutions

1. Use one of the accepted tokens: list, read, search, grep, edit, patch, apply_patch, bash, shell, exec
2. For the apply-patch step the token is apply_patch or patch — underscore, no hyphen
3. Run codewhale eval --help to re-check the flag after upgrading
4. If a new step kind is genuinely needed, extend ScenarioStepKind in crates/tui/src/eval.rs

Example fix

# before
codewhale eval --fail-step apply-patch

# after
codewhale eval --fail-step apply_patch

const VALID_FAIL_STEPS: &[&str] = &[
    "list", "read", "search", "grep", "edit",
    "patch", "apply_patch", "bash", "shell", "exec",
];
if let Some(value) = &args.fail_step {
    let ok = VALID_FAIL_STEPS.contains(&value.trim().to_lowercase().as_str());
    assert!(ok, "invalid --fail-step '{value}'");
}

fn is_valid_fail_step(value: &str) -> bool {
    matches!(
        value.trim().to_lowercase().as_str(),
        "list" | "read" | "search" | "grep" | "edit"
            | "patch" | "apply_patch" | "bash" | "shell" | "exec"
    )
}

When it happens

Trigger: Passing a token outside the accepted set — for example --fail-step write, --fail-step file (the agent tool name, not a step kind), or --fail-step apply-patch (hyphen instead of underscore).

Common situations: Assuming the flag takes any tool name; hyphenation differences (apply-patch vs apply_patch); misspellings. Case and trailing spaces are tolerated.

Related errors

• No local {} API key was found in config, the secret store, o
• Codewhale account API base URL must include a host
• The Codewhale service returned a verification URL without a
• Codewhale account request failed (HTTP {}, code {code})
• Codewhale account request failed (HTTP {})

AI-assisted analysis of Hmbown/CodeWhale@8880682c63 (2026-08-16). Data as JSON: /api/errors/3e965a32a8ffc0f9. Report an issue: GitHub.