{"record":{"id":"e2e7309526c1c974","repo":"github/scientist","slug":"experiment-name-observations-mismatched","errorCode":null,"errorMessage":"experiment '#{name}' observations mismatched","messagePattern":"experiment '#(.+?)' observations mismatched","errorType":"exception","errorClass":"Scientist::Experiment::MismatchError","httpStatus":null,"severity":"error","filePath":"lib/scientist/experiment.rb","lineNumber":257,"sourceCode":"    end\n\n    result = generate_result(name)\n\n    if @_scientist_after_run\n      @_scientist_after_run.call(result)\n    end\n\n    begin\n      publish(result)\n    rescue StandardError => ex\n      raised :publish, ex\n    end\n\n    if raise_on_mismatches? && result.mismatched?\n      if @_scientist_custom_mismatch_error\n        raise @_scientist_custom_mismatch_error.new(self.name, result)\n      else\n        raise MismatchError.new(self.name, result)\n      end\n    end\n\n    control = result.control\n    raise control.exception if control.raised?\n    control.value\n  end\n\n  # Define a block that determines whether or not the experiment should run.\n  def run_if(&block)\n    @_scientist_run_if_block = block\n  end\n\n  # Internal: does a run_if block allow the experiment to run?\n  #\n  # Rescues and reports exceptions in a run_if block if they occur.\n  def run_if_block_allows?\n    (@_scientist_run_if_block ? @_scientist_run_if_block.call : true)","sourceCodeStart":239,"sourceCodeEnd":275,"githubUrl":"https://github.com/github/scientist/blob/504a396e987f655a21c6bf2ed57935aadaa40859/lib/scientist/experiment.rb#L239-L275","documentation":"Scientist::Experiment::MismatchError is raised by Experiment#run when `raise_on_mismatches?` is enabled and the published result has mismatched observations (candidate returned a different value, or raised a different error, than control per the configured comparators). If a custom error class was set via `raise_with`, that class is instantiated and raised instead.","triggerScenarios":"Calling `experiment.run` with raise_on_mismatches enabled (via `raise_on_mismatches { true }`) when the candidate block returns a value not equivalent to control, or when a `compare`/`compare_errors` lambda reports inequality. Also raised when the custom `raise_with` class is instantiated on a mismatch.","commonSituations":"Test/CI environments that enable raise_on_mismatches to catch divergent behavior; candidate and control genuinely diverging after a code change; an overly strict comparator (timestamps, object identity) causing false mismatches; candidate code upgraded to a different return shape.","solutions":["Fix the underlying behavioral difference so candidate and control return equivalent results.","Relax or correct the `compare` / `compare_errors` comparator if the mismatch is a false positive.","Disable raising on mismatches (`raise_on_mismatches { false }`) and inspect published results instead.","Rescue Scientist::Experiment::MismatchError (or your raise_with class) around run if divergence is tolerable."],"exampleFix":"# before\nexperiment.raise_on_mismatches { true }\nexperiment.use { user.name }\nexperiment.try { user.full_name } # mismatch raised in run\n\n# after\nexperiment.compare { |a, b| a.to_s == b.to_s }\nexperiment.raise_on_mismatches { Rails.env.test? }\nexperiment.use { user.name }\nexperiment.try { user.full_name }","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"begin\n  value = experiment.run\nrescue Scientist::Experiment::MismatchError => e\n  metrics.increment(\"scientist.mismatch\")\n  e.result.control.value # fall back to control behavior\nend","preventionTips":["Enable raise_on_mismatches only in test/CI; keep it off in production paths.","Write explicit `compare` lambdas covering known non-determinism (time, ids, ordering).","Use `compare_errors` so differing-but-acceptable exception classes don't count as mismatches.","Watch published results in your publisher to catch drift before it raises."],"tags":["ruby","scientist","mismatch","a-b-testing"],"backgroundTag":"unexpected-response-shape","analyzedSha":"504a396e987f655a21c6bf2ed57935aadaa40859","analyzedAt":"2026-09-13T22:55:19.059Z","contentChangedAt":"2026-09-13T22:55:19.059Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}