presidentbeef/brakeman · error · ArgumentError

Comparison file doesn't exist

Error message

Comparison file doesn't exist

What it means

Brakeman's comparison mode (CLI flag `--compare FILE`) re-runs a scan and diffs it against a previously saved JSON report. Before doing anything, `Brakeman.compare` checks `File.exist?(options[:previous_results_json])` and raises `ArgumentError` if the baseline file is not on disk. This is a pre-flight guard: the comparison is impossible without a prior JSON report.

Source

Thrown at lib/brakeman.rb:556

  end

  def self.announce message
    logger.announce message
  end

  def self.alert message
    logger.alert message
  end

  def self.debug message
    logger.debug message
  end

  # Compare JSON output from a previous scan and return the diff of the two scans
  def self.compare options
    require 'json'
    require 'brakeman/differ'
    raise ArgumentError.new("Comparison file doesn't exist") unless File.exist? options[:previous_results_json]

    begin
      previous_results = JSON.parse(File.read(options[:previous_results_json]), :symbolize_names => true)[:warnings]
    rescue JSON::ParserError
      self.alert "Error parsing comparison file: #{options[:previous_results_json]}"
      exit!
    end

    tracker = run(options)
    new_report = JSON.parse(tracker.report.to_json, symbolize_names: true)

    new_results = new_report[:warnings]
    obsolete_ignored = tracker.unused_fingerprints

    Brakeman::Differ.new(new_results, previous_results).diff.tap do |diff|
      diff[:obsolete] = obsolete_ignored
    end
  end

View on GitHub (pinned to 649e678d0a)

Solutions

1. Confirm the file actually exists at the exact path passed: `ls -l tmp/brakeman.json` from the same directory brakeman runs in.
2. If you have no baseline yet, generate one first: `brakeman -o tmp/brakeman.json`, then run `brakeman --compare tmp/brakeman.json`.
3. Use an absolute path (e.g. `--compare $PWD/tmp/brakeman.json`) or run brakeman from the directory the relative path is written against.
4. In CI, make the compare step conditional: restore the baseline artifact and only call `--compare` when it exists; otherwise run a normal scan that produces the baseline for the next run.

Example fix

# before (first CI run, baseline does not exist yet)
brakeman --compare tmp/brakeman.json   # => ArgumentError: Comparison file doesn't exist

# after (generate baseline, then compare)
brakeman -o tmp/brakeman.json
brakeman --compare tmp/brakeman.json

# Ruby, before invoking Brakeman.compare / `brakeman --compare`
baseline = Rails.root.join('tmp/brakeman.json')
if File.exist?(baseline)
  Brakeman.compare :previous_results_json => baseline.to_s, :app_path => Rails.root.to_s
else
  warn 'No baseline report found; generating one now.'
  Brakeman.run(:app_path => Rails.root.to_s, :output_files => [baseline.to_s])
end

begin
  Brakeman.compare :previous_results_json => path, :app_path => app
rescue ArgumentError => e
  abort "#{e.message} — run `brakeman -o #{path}` first to create the baseline."
end

When it happens

Trigger: Running `brakeman --compare tmp/brakeman.json` (or programmatically `Brakeman.compare :previous_results_json => path`) when that path does not exist: the baseline was never generated with `-o`/`--output`, the relative path resolves from a different working directory, the filename has a typo, or a CI artifact from a previous job was not restored.

Common situations: CI pipelines that run `--compare` on the very first build (no baseline artifact yet); baseline saved with a different filename than the one passed to `--compare`; artifact uploaded to a job that isn't downloaded before the compare step; running brakeman from a subdirectory so a relative path like `tmp/brakeman.json` points elsewhere.

Related errors

• Cannot specify output format if multiple output files specif
• Invalid GitHub repository format
• Could not find specified check#{missing.length > 1 ? 's' : '
• Minimum age must be 1-15 days.
• Invalid format options: #{invalid_options.inspect}

AI-assisted analysis of presidentbeef/brakeman@649e678d0a (2026-08-21). Data as JSON: /api/errors/84a2ac892d849776. Report an issue: GitHub.