rtk-ai/rtk · error · anyhow::Error

{} test(s) failed

Error message

{} test(s) failed

What it means

rtk verify applies each filter to its inline test inputs and compares the transformed output against the declared `expected` value. A nonzero failure count exits with '{n} test(s) failed'; the filters themselves ran, but at least one no longer transforms its fixture as specified. Each failure is printed above with filter name, test name, and expected vs actual.

Source

Thrown at src/hooks/verify_cmd.rs:45

    if total == 0 {
        println!("No inline tests found.");
    } else {
        println!("{}/{} tests passed", passed, total);
    }

    if require_all && !results.filters_without_tests.is_empty() {
        for name in &results.filters_without_tests {
            eprintln!("MISSING tests for filter: {}", name);
        }
        anyhow::bail!(
            "{} filter(s) have no inline tests (use --require-all in CI)",
            results.filters_without_tests.len()
        );
    }

    if failed > 0 {
        anyhow::bail!("{} test(s) failed", failed);
    }

    Ok(())
}

View on GitHub (pinned to d977e1c316)

Solutions

1. Read the per-test 'FAIL [filter] name / expected / actual' block printed above the bail
2. If the new actual output is the intended behavior, update the inline expected strings; otherwise fix the filter rules
3. Re-run `rtk verify` until the summary shows 0 failed before merging

Example fix

# .rtk/filters.toml — regex tightened, fixture stale
[[tests.make]]
input = "make[1]: Entering directory '/x'\ngcc foo.c\n"
expected = "make[1]: Entering directory '/x'\ngcc foo.c\n"   # stale

# updated to the new intended behavior
expected = "gcc foo.c\n"

# run inline tests locally before pushing filter changes
rtk verify

In CI, parse the 'FAIL [<filter>] <test>' stderr blocks: fail the job, print the expected/actual pair, and never auto-update `expected` — a human must decide whether the filter or the fixture is right.

When it happens

Trigger: Editing a filter's patterns (match_output/strip_lines_matching/replace rules) without updating its inline `expected` strings — or vice versa — then running `rtk verify`; bail at src/hooks/verify_cmd.rs:47.

Common situations: Tightening a regex fixes one command but breaks another fixture; a formatting change (headers, ellipsis, grouping) invalidates many expectations at once.

Related errors

• {} filter(s) have no inline tests (use --require-all in CI)
• Unknown filter '{}'. Available: cargo-test, pytest, go-test,
• Invalid hash format in {} (expected 'hash filename')
• Invalid SHA-256 hash in {}
• Filter file present but not valid TOML — see the error above

AI-assisted analysis of rtk-ai/rtk@d977e1c316 (2026-08-16). Data as JSON: /api/errors/a4acfc9b6ba3844b. Report an issue: GitHub.