we-promise/sure · error · ActiveRecord::RecordInvalid

duplicate_headers

duplicate_headers

Error message

CSV headers normalize to duplicate columns: %{columns}

What it means

During CSV header normalization, Import strips and downcases headers, removes '*', and collapses spaces/dashes to underscores (normalize_header). If distinct original headers collapse to the same normalized name ('Date' vs 'date*', 'Transaction Amount' vs 'transaction-amount'), the validation adds :duplicate_headers to :base with the offending column lists and raises ActiveRecord::RecordInvalid, because column mapping must stay unambiguous.

Solutions

1. Open the CSV and rename the headers listed in the error's columns text so each is unique after normalization
2. Remove decorative '*' suffixes and case variants before upload
3. If two columns are legitimately identically named, delete or rename one — the importer cannot guess which column maps where

Example fix

// before
import.publish_later # upload already failed validation: duplicate_headers

// after — check before upload
headers = CSV.open(file.path, headers: true).read.headers.to_a.compact
normalized = headers.map { |h| h.to_s.strip.downcase.gsub(/\*/, '').gsub(/[\s-]+/, '_') }
raise 'CSV headers normalize to duplicates' unless normalized.uniq.length == normalized.length

headers = CSV.open(file.path, headers: true).read.headers.to_a.compact
normalized = headers.map { |h| h.to_s.strip.downcase.gsub(/\*/, '').gsub(/[\s-]+/, '_') }
normalized.uniq.length == normalized.length # false => the model will raise ActiveRecord::RecordInvalid with :duplicate_headers

rescue ActiveRecord::RecordInvalid => e near the upload flow and read record.errors.details[:base] for the :duplicate_headers code with the offending column lists; show the lists verbatim so the user knows which headers to rename

When it happens

Trigger: Uploading a CSV whose headers differ only by case, whitespace, asterisks, or dash/underscore separators — normalization maps both originals to one column key, and grouped_headers detects a normalized name with multiple originals.

Common situations: Spreadsheet exports that decorate duplicate headers ('Amount', 'AMOUNT', 'Amount*'); concatenated or hand-edited CSVs; exports whose second same-named column is a notes/total column.

Related errors

• content_too_large
• file_too_large
• invalid_file_type
• max_row_count_exceeded
• max_row_count_exceeded

AI-assisted analysis of we-promise/sure@e69894adb9 (2026-08-21). Data as JSON: /api/errors/d07d05e73d8a340f. Report an issue: GitHub.

Appendix: source

Thrown at app/models/import.rb:602

    def normalized_csv_headers
      @normalized_csv_headers ||= begin
        grouped_headers = Array(csv_headers)
          .filter_map do |header|
            normalized = normalize_header(header)
            next if normalized.blank?

            [ normalized, header ]
          end
          .group_by(&:first)

        duplicate_headers = grouped_headers.values.filter_map do |headers|
          originals = headers.map(&:last).uniq
          originals if originals.many?
        end

        if duplicate_headers.any?
          errors.add(:base, :duplicate_headers, columns: duplicate_headers.map { |headers| headers.join(", ") }.join("; "))
          raise ActiveRecord::RecordInvalid, self
        end

        grouped_headers.transform_values { |headers| headers.first.last }
      end
    end

    def normalize_header(header)
      header.to_s.strip.downcase.gsub(/\*/, "").gsub(/[\s-]+/, "_")
    end

    def parsed_csv
      return @parsed_csv if defined?(@parsed_csv)

      csv_content = raw_file_str || ""
      if rows_to_skip.to_i > 0
        csv_content = csv_content.lines.drop(rows_to_skip).join
      end

View on GitHub (pinned to e69894adb9)