{"record":{"id":"e2594b39e92c2b2e","repo":"Freika/dawarich","slug":"could-not-detect-required-columns-latitude-longi","errorCode":null,"errorMessage":"Could not detect required columns: latitude, longitude, timestamp. Found headers must include recognized aliases for all three.","messagePattern":"Could not detect required columns: latitude, longitude, timestamp\\. Found headers must include recognized aliases for all three\\.","errorType":"exception","errorClass":"Csv::Detector::DetectionError","httpStatus":null,"severity":"error","filePath":"app/services/csv/detector.rb","lineNumber":91,"sourceCode":"\n      # Fallback: substring matching for non-standard headers like \"LATITUDE N/S\"\n      columns[:latitude]  ||= find_header_by_substring(headers, 'latitude')\n      columns[:longitude] ||= find_header_by_substring(headers, 'longitude')\n      columns[:timestamp] ||= find_header_by_substring(headers, 'timestamp')\n\n      columns\n    end\n\n    def find_header_by_substring(headers, keyword)\n      normalized = headers.map { |h| h.to_s.downcase.strip }\n      normalized.index { |h| h.include?(keyword) }\n    end\n\n    def validate_columns!(columns)\n      missing = %i[latitude longitude timestamp].reject { |f| columns[f] }\n      return if missing.empty?\n\n      raise DetectionError,\n            I18n.t('services.csv.detector.required_columns_missing')\n    end\n\n    def parse_data_rows(lines, delimiter)\n      lines.first(DATA_SAMPLE_SIZE).filter_map do |line|\n        row = CSV.parse_line(line, col_sep: delimiter)&.map { |v| v&.strip }\n        row if row&.any?(&:present?)\n      end\n    end\n\n    def detect_coordinate_format(data_rows, columns)\n      lat_idx = columns[:latitude]\n      lon_idx = columns[:longitude]\n      return :decimal_degrees unless lat_idx && lon_idx\n\n      lat_values = data_rows.filter_map { |row| row[lat_idx] }\n      lon_values = data_rows.filter_map { |row| row[lon_idx] }\n      all_coords = lat_values + lon_values","sourceCodeStart":73,"sourceCodeEnd":109,"githubUrl":"https://github.com/Freika/dawarich/blob/97fad417c5a11b0eb11157890635e015723a2e97/app/services/csv/detector.rb#L73-L109","documentation":"Raised as Csv::Detector::DetectionError when the detector cannot map the sample's headers onto the three mandatory columns: latitude, longitude, timestamp. Headers are matched first via Imports::FieldAliases (exact recognized aliases) then a downcased substring fallback ('latitude', 'longitude', 'timestamp'); if any of the three still has no column index, validation fails. It fires during import setup, before any rows are parsed.","triggerScenarios":"A CSV whose headers use unrecognized names for the required fields (e.g. 'lat_dd'/'lon_dd' with no 'latitude' substring, 'time' instead of a 'timestamp'-containing header), a first data row mistaken for headers (headerless CSV), a wrong delimiter guess that merges all headers into one cell, or headers hidden behind a BOM/quotes that break parsing.","commonSituations":"Users exporting from a tracker app whose column names differ (e.g. 'Latitude N/S' works via substring but 'position_lat' does not), Excel exports saving semicolon-delimited while detection picked comma, Ukrainian/localized header names, a JSON or KML file mislabeled .csv.","solutions":["Open the file and confirm row 1 contains headers including recognized aliases for latitude, longitude, and timestamp (e.g. 'latitude'/'lat', 'longitude'/'lon'/'lng', 'timestamp'/'time' as aliased in Imports::FieldAliases).","Rename the offending columns to standard names (Latitude, Longitude, Timestamp) — the substring fallback accepts anything containing 'latitude'/'longitude'/'timestamp' after downcasing.","If the file has no header row, add one; if the delimiter is ; or tab, keep it consistent on every line so delimiter detection locks on.","Check for a mangled first line (BOM without utf-8 handling, stray quotes) by parsing the header in a console."],"exampleFix":"# before (file headers)\n# position_lat,position_lon,time\n# -> DetectionError: required columns missing\n\n# after (rename headers)\n# latitude,longitude,timestamp\n# 52.5200,13.4050,2024-05-01T10:00:00Z","handlingStrategy":"validation","validationCode":"sample = File.foreach(path, chomp: true).lazy.reject(&:empty?).first(12)\nheaders = CSV.parse_line(sample.first || '')&.map { |h| h.to_s.downcase.strip } || []\n%w[latitude longitude timestamp].all? { |k| headers.any? { |h| h.include?(k) } } || raise('headers missing required columns')","typeGuard":"def csv_with_required_headers?(path)\n  header = CSV.parse_line(File.foreach(path, chomp: true).next || '')\n  norm = (header || []).map { |h| h.to_s.downcase.strip }\n  %w[latitude longitude timestamp].all? { |k| norm.any? { |h| h.include?(k) } }\nrescue StandardError\n  false\nend","tryCatchPattern":"begin\n  Csv::Detector.new(path).call\nrescue Csv::Detector::DetectionError => e\n  import.update!(status: :failed, error_message: 'CSV must contain columns recognizable as latitude, longitude, and timestamp')\nend","preventionTips":["Template/normalize exports to headers named Latitude, Longitude, Timestamp before upload.","Show users which header aliases are accepted on the import UI.","Unit-test the detector with your real tracker's export headers."],"tags":["csv","import","header-validation","data-format","ruby"],"backgroundTag":"csv-header-validation-failed","analyzedSha":"97fad417c5a11b0eb11157890635e015723a2e97","analyzedAt":"2026-08-21T17:04:17.778Z","schemaVersion":2},"datasetVersion":"2026-08-21T18:17:14.833Z"}