What causes "{} test(s) failed" in rtk-ai/rtk?

Read the FAIL blocks on stderr: compare each test's `expected` vs `actual`, then fix the filter pattern or update the test to the correct expectation.

rtk-ai/rtk · error

test(s) failed

Error message

{} test(s) failed

What it means

`rtk verify` executes the inline tests defined inside TOML custom filters. If any executed test fails (its actual filtered output does not match the expected output), the command prints each failure with expected/actual on stderr, reports the pass count, then bails with the total number of failing tests. It means one or more filter rules no longer behave as their tests specify.

Solutions

Read the FAIL blocks on stderr: compare each test's `expected` vs `actual`, then fix the filter pattern or update the test to the correct expectation.
Re-run `rtk verify <filter-name>` scoped to the failing filter for a faster iterate loop.
If the new behavior is correct, update the inline test's `expected` value; if not, revert the filter change.
Run `rtk verify` locally as part of the pre-commit gate so failures are caught before CI.

Example fix

// before: test expectation out of sync with the pattern
[[filters.my-filter.tests]]
input = "warning: slow"
expected = "SLOW"

// after: correct expectation after pattern change
[[filters.my-filter.tests]]
input = "warning: slow"
expected = "slow"

Defensive patterns

Strategy: try-catch

Try / catch

import subprocess
r = subprocess.run(["rtk", "verify"], capture_output=True, text=True)
if r.returncode != 0 and "test(s) failed" in r.stderr:
    for line in r.stderr.splitlines():
        if line.startswith("FAIL"):
            print(line)  # surface expected vs actual to the developer
    raise SystemExit("fix filter patterns or update inline test expectations")

Prevention

Run `rtk verify` locally after every edit to a filter's pattern or replacement.
When changing a filter intentionally, update its inline expected values in the same commit.
Keep test inputs small and deterministic so expected outputs are easy to maintain.
Wire `rtk verify` into the pre-commit gate so failures never reach CI.

When it happens

Trigger: Running `rtk verify` (with or without --require-all, and with or without a filter-name argument) when `run_filter_tests` returns outcomes where `passed == false` for at least one test — i.e. failed > 0 after the require-all check.

Common situations: Editing a filter's pattern or replacement without updating its inline tests; a test whose expected output was written for an older filter format; typos in expected strings; a regex change that alters which lines match.

Related errors

AI-assisted analysis of rtk-ai/rtk@0924356b4c (2026-09-19). Data as JSON: /api/errors/a4acfc9b6ba3844b. Report an issue: GitHub.

Appendix: source

Thrown at src/hooks/verify_cmd.rs:45

    if total == 0 {
        println!("No inline tests found.");
    } else {
        println!("{}/{} tests passed", passed, total);
    }

    if require_all && !results.filters_without_tests.is_empty() {
        for name in &results.filters_without_tests {
            eprintln!("MISSING tests for filter: {}", name);
        }
        anyhow::bail!(
            "{} filter(s) have no inline tests (use --require-all in CI)",
            results.filters_without_tests.len()
        );
    }

    if failed > 0 {
        anyhow::bail!("{} test(s) failed", failed);
    }

    Ok(())
}

View on GitHub (pinned to 0924356b4c)