we-promise/sure · error · ActiveRecord::RecordInvalid

duplicate_headers

duplicate_headers

Error message

CSV headers normalize to duplicate columns: %{columns}

What it means

During CSV header normalization, Import strips and downcases headers, removes '*', and collapses spaces/dashes to underscores (normalize_header). If distinct original headers collapse to the same normalized name ('Date' vs 'date*', 'Transaction Amount' vs 'transaction-amount'), the validation adds :duplicate_headers to :base with the offending column lists and raises ActiveRecord::RecordInvalid, because column mapping must stay unambiguous.

Source

Thrown at app/models/import.rb:602

    def normalized_csv_headers
      @normalized_csv_headers ||= begin
        grouped_headers = Array(csv_headers)
          .filter_map do |header|
            normalized = normalize_header(header)
            next if normalized.blank?

            [ normalized, header ]
          end
          .group_by(&:first)

        duplicate_headers = grouped_headers.values.filter_map do |headers|
          originals = headers.map(&:last).uniq
          originals if originals.many?
        end

        if duplicate_headers.any?
          errors.add(:base, :duplicate_headers, columns: duplicate_headers.map { |headers| headers.join(", ") }.join("; "))
          raise ActiveRecord::RecordInvalid, self
        end

        grouped_headers.transform_values { |headers| headers.first.last }
      end
    end

    def normalize_header(header)
      header.to_s.strip.downcase.gsub(/\*/, "").gsub(/[\s-]+/, "_")
    end

    def parsed_csv
      return @parsed_csv if defined?(@parsed_csv)

      csv_content = raw_file_str || ""
      if rows_to_skip.to_i > 0
        csv_content = csv_content.lines.drop(rows_to_skip).join
      end

View on GitHub (pinned to e69894adb9)

Solutions

1. Open the CSV and rename the headers listed in the error's columns text so each is unique after normalization
2. Remove decorative '*' suffixes and case variants before upload
3. If two columns are legitimately identically named, delete or rename one — the importer cannot guess which column maps where

Example fix

// before
import.publish_later # upload already failed validation: duplicate_headers

// after — check before upload
headers = CSV.open(file.path, headers: true).read.headers.to_a.compact
normalized = headers.map { |h| h.to_s.strip.downcase.gsub(/\*/, '').gsub(/[\s-]+/, '_') }
raise 'CSV headers normalize to duplicates' unless normalized.uniq.length == normalized.length

headers = CSV.open(file.path, headers: true).read.headers.to_a.compact
normalized = headers.map { |h| h.to_s.strip.downcase.gsub(/\*/, '').gsub(/[\s-]+/, '_') }
normalized.uniq.length == normalized.length # false => the model will raise ActiveRecord::RecordInvalid with :duplicate_headers

rescue ActiveRecord::RecordInvalid => e near the upload flow and read record.errors.details[:base] for the :duplicate_headers code with the offending column lists; show the lists verbatim so the user knows which headers to rename

When it happens

Trigger: Uploading a CSV whose headers differ only by case, whitespace, asterisks, or dash/underscore separators — normalization maps both originals to one column key, and grouped_headers detects a normalized name with multiple originals.

Common situations: Spreadsheet exports that decorate duplicate headers ('Amount', 'AMOUNT', 'Amount*'); concatenated or hand-edited CSVs; exports whose second same-named column is a notes/total column.

Related errors

• max_row_count_exceeded
• file_too_large
• content_too_large
• invalid_file_type
• max_row_count_exceeded

AI-assisted analysis of we-promise/sure@e69894adb9 (2026-08-21). Data as JSON: /api/errors/d07d05e73d8a340f. Report an issue: GitHub.