{"record":{"id":"5249218c57624e80","repo":"minitest/minitest","slug":"reproduction-failed-not-false-positive-aborting","errorCode":null,"errorMessage":"Reproduction failed? Not false positive. Aborting.","messagePattern":"Reproduction failed\\? Not false positive\\. Aborting\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"lib/minitest/bisect.rb","lineNumber":183,"sourceCode":"    normal   = bad_names.empty?\n    inverted = !normal\n\n    if inverted then\n      time_it \"reproducing w/ scoped failure (inverted run!)...\", build_methods_cmd(build_files_cmd(files, rb_flags, mt_flags + bad_names))\n      raise \"No failures. Probably not a false positive. Aborting.\" if failures.empty?\n      bad = map_failures\n    end\n\n    cmd = build_files_cmd(files, rb_flags, mt_flags)\n\n    msg = normal ? \"reproducing...\" : \"reproducing false positive...\"\n    time_it msg, build_methods_cmd(cmd)\n\n    if normal then\n      raise \"Reproduction run passed? Aborting.\" unless tainted?\n      raise \"Verification failed. No culprits? Aborting.\" if culprits.empty? && seen_bad\n    else\n      raise \"Reproduction failed? Not false positive. Aborting.\" if tainted?\n      raise \"Verification failed. No culprits? Aborting.\" if culprits.empty? || seen_bad\n    end\n\n    if normal then\n      bad = map_failures\n\n      time_it \"verifying...\", build_methods_cmd(cmd, [], bad)\n\n      new_bad = map_failures\n\n      if bad == new_bad then\n        warn \"Tests fail by themselves. This may not be an ordering issue.\"\n      end\n    end\n\n    idx = culprits.index bad.first\n    self.culprits = culprits.take idx+1 if idx # cull tests after bad\n","sourceCodeStart":165,"sourceCodeEnd":201,"githubUrl":"https://github.com/minitest/minitest/blob/581e7d5386def72040e49b036ba49e36e9cb14df/lib/minitest/bisect.rb#L165-L201","documentation":"In inverted mode (with -n/--name), after step 1 confirms the scoped run fails, bisect verifies the false-positive hypothesis by running the whole suite ('reproducing false positive...', bisect.rb:177). If that full run is also tainted, the test fails with everything too, so it is a genuine failure rather than a false positive, and bisect aborts with 'Reproduction failed? Not false positive. Aborting.' (bisect.rb:183).","triggerScenarios":"minitest_bisect -s 1234 -n '/test_foo/' ... where the test fails both scoped AND in the full run — the test is simply broken, or the chosen seed reproduces the failure globally.","commonSituations":"Reaching for inverted mode by mistake (it answers 'what makes this failing test pass'); a real failure first noticed in a scoped run; misreading which mode the -n filter selects.","solutions":["Use normal mode instead — drop the -n filter: minitest_bisect -s <seed> test/**/*_test.rb — to find the minimal failing combination","If the test fails by itself, stop bisecting and debug it directly","Confirm both runs with MTB_VERBOSE=2 to see the exact commands bisect executed and their pass/fail behavior","Double-check the intent: inverted mode is only for tests that fail scoped but pass with the full suite"],"exampleFix":"# before\nminitest_bisect -s 424242 -n '/test_paypal/' test/**/*_test.rb   # aborts: not a false positive\n\n# after\nminitest_bisect -s 424242 test/**/*_test.rb                     # normal mode: minimal failing combo","handlingStrategy":"validation","validationCode":"# inverted mode is only valid when scoped fails AND full run passes:\nseed = 424242\nscoped_fails = !system(\"ruby -Itest test/**/*_test.rb -s #{seed} -n /test_paypal/\")\nfull_passes  =  system(\"ruby -Itest test/**/*_test.rb -s #{seed}\")\nputs \"use normal bisect (drop -n)\" unless scoped_fails && full_passes","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Know the modes: normal finds a minimal failing combo; inverted (-n) finds what makes a scoped failing test pass","Verify the pass/fail shape of both runs (scoped and full) with the same seed before invoking bisect","If the test fails everywhere, debug it directly — no bisection mode applies","Use MTB_VERBOSE=2 to watch the reproduction runs bisect performs"],"tags":["minitest","bisect","test-order","false-positive","ruby"],"backgroundTag":"test-order-dependency","analyzedSha":"581e7d5386def72040e49b036ba49e36e9cb14df","analyzedAt":"2026-08-23T11:44:02.076Z","schemaVersion":2},"datasetVersion":"2026-08-23T16:17:53.355Z"}