minitest/minitest · error · RuntimeError

Reproduction failed? Not false positive. Aborting.

Error message

Reproduction failed? Not false positive. Aborting.

What it means

In inverted mode (with -n/--name), after step 1 confirms the scoped run fails, bisect verifies the false-positive hypothesis by running the whole suite ('reproducing false positive...', bisect.rb:177). If that full run is also tainted, the test fails with everything too, so it is a genuine failure rather than a false positive, and bisect aborts with 'Reproduction failed? Not false positive. Aborting.' (bisect.rb:183).

Source

Thrown at lib/minitest/bisect.rb:183

    normal   = bad_names.empty?
    inverted = !normal

    if inverted then
      time_it "reproducing w/ scoped failure (inverted run!)...", build_methods_cmd(build_files_cmd(files, rb_flags, mt_flags + bad_names))
      raise "No failures. Probably not a false positive. Aborting." if failures.empty?
      bad = map_failures
    end

    cmd = build_files_cmd(files, rb_flags, mt_flags)

    msg = normal ? "reproducing..." : "reproducing false positive..."
    time_it msg, build_methods_cmd(cmd)

    if normal then
      raise "Reproduction run passed? Aborting." unless tainted?
      raise "Verification failed. No culprits? Aborting." if culprits.empty? && seen_bad
    else
      raise "Reproduction failed? Not false positive. Aborting." if tainted?
      raise "Verification failed. No culprits? Aborting." if culprits.empty? || seen_bad
    end

    if normal then
      bad = map_failures

      time_it "verifying...", build_methods_cmd(cmd, [], bad)

      new_bad = map_failures

      if bad == new_bad then
        warn "Tests fail by themselves. This may not be an ordering issue."
      end
    end

    idx = culprits.index bad.first
    self.culprits = culprits.take idx+1 if idx # cull tests after bad

View on GitHub (pinned to 581e7d5386)

Solutions

1. Use normal mode instead — drop the -n filter: minitest_bisect -s <seed> test/**/*_test.rb — to find the minimal failing combination
2. If the test fails by itself, stop bisecting and debug it directly
3. Confirm both runs with MTB_VERBOSE=2 to see the exact commands bisect executed and their pass/fail behavior
4. Double-check the intent: inverted mode is only for tests that fail scoped but pass with the full suite

Example fix

# before
minitest_bisect -s 424242 -n '/test_paypal/' test/**/*_test.rb   # aborts: not a false positive

# after
minitest_bisect -s 424242 test/**/*_test.rb                     # normal mode: minimal failing combo

# inverted mode is only valid when scoped fails AND full run passes:
seed = 424242
scoped_fails = !system("ruby -Itest test/**/*_test.rb -s #{seed} -n /test_paypal/")
full_passes  =  system("ruby -Itest test/**/*_test.rb -s #{seed}")
puts "use normal bisect (drop -n)" unless scoped_fails && full_passes

When it happens

Trigger: minitest_bisect -s 1234 -n '/test_foo/' ... where the test fails both scoped AND in the full run — the test is simply broken, or the chosen seed reproduces the failure globally.

Common situations: Reaching for inverted mode by mistake (it answers 'what makes this failing test pass'); a real failure first noticed in a scoped run; misreading which mode the -n filter selects.

Related errors

• No failures. Probably not a false positive. Aborting.
• Verification failed. No culprits? Aborting.
• Reproduction run passed? Aborting.
• class or module required for rescue clause. Got %p
• Skipped, no message given

AI-assisted analysis of minitest/minitest@581e7d5386 (2026-08-23). Data as JSON: /api/errors/5249218c57624e80. Report an issue: GitHub.