Hmbown/CodeWhale · error

invalid --fail-step

Error message

invalid --fail-step '{value}'

What it means

Thrown by `run_eval` when the `--fail-step` CLI argument cannot be parsed into a `ScenarioStepKind` via `ScenarioStepKind::parse`. The offline evaluation harness only accepts known step-kind names for this flag; anything else aborts before the harness runs.

Solutions

1. Run `codewhale eval --help` (or the equivalent) to list valid --fail-step values.
2. Correct the spelling/case of the step kind exactly as `ScenarioStepKind::parse` expects.
3. Omit --fail-step entirely if you do not need to force a specific step failure.

Example fix

// before
codewhale eval --fail-step toolcall
// after
codewhale eval --fail-step tool

// check the value against accepted step kinds before running
let ok = ["tool", "model", /* ... */].contains(&value);
if !ok { eprintln!("invalid --fail-step '{value}'"); std::process::exit(2); }

When it happens

Trigger: Running the eval subcommand with `--fail-step` set to a string that is not a recognized scenario step kind, e.g. a typo (`--fail-step tool_use` when the kind is `tool`), wrong case, or an empty value.

Common situations: Copy-pasting a step name from docs of a different version; guessing step kinds without checking `--help`; shell quoting strips or mangles the value.

Understand the failure class

Background: "Unknown argument", "Invalid value", and "must be one of": invalid CLI argument errors explained — this error's family across 35 libraries.

Related errors

• --continue/--resume/--session-id apply to the interactive…
• unknown argument
• unknown argument
• unknown argument
• Unknown or empty option. Use --help.

AI-assisted analysis of Hmbown/CodeWhale@73e0f67d83 (2026-09-22). Data as JSON: /api/errors/3e965a32a8ffc0f9. Report an issue: GitHub.

Appendix: source

Thrown at crates/tui/src/lib.rs:3075

        );
    }
    std::fs::File::open(path)
        .map_err(|error| anyhow!("could not securely open {}: {error}", path.display()))
}

/// Generate shell completions for the given shell
fn generate_completions(shell: Shell) {
    let mut cmd = Cli::command();
    let name = cmd.get_name().to_string();
    generate(shell, &mut cmd, name, &mut io::stdout());
}

/// Run the offline evaluation harness (no network/LLM calls).
fn run_eval(args: EvalArgs) -> Result<()> {
    let fail_step = match args.fail_step.as_deref() {
        Some(value) => ScenarioStepKind::parse(value)
            .map(Some)
            .ok_or_else(|| anyhow!("invalid --fail-step '{value}'"))?,
        None => None,
    };

    let config = EvalHarnessConfig {
        fail_step,
        shell_command: args.shell_command,
        shell_expect_token: args.shell_expect_token,
        max_output_chars: args.max_output_chars,
        record_dir: args.record.clone(),
        ..EvalHarnessConfig::default()
    };

    let harness = EvalHarness::new(config);
    let run = harness.run().context("evaluation harness failed")?;
    let report = run.to_report();

    if args.json {
        let json = serde_json::to_string_pretty(&report)?;

View on GitHub (pinned to 73e0f67d83)