instructure/canvas-lms · error · ParseError

Not a valid utf-8 string

Error message

Not a valid utf-8 string: %{string}

What it means

Raised by Outcomes::CsvImporter#check_encoding when a string read from the CSV file is not valid UTF-8. The importer force-encodes the input to UTF-8 and calls valid_encoding?; if the byte sequence cannot be a valid UTF-8 string (e.g. Latin-1 or Windows-1252 bytes), it aborts the parse with a ParseError. It exists to fail fast before rows are parsed with an unrepresentable encoding.

Solutions

1. Re-save/re-encode the CSV as UTF-8 (e.g. `iconv -f WINDOWS-1252 -t UTF-8 input.csv > output.csv` or export as 'CSV UTF-8' from Excel).
2. Detect the actual encoding (`file -i input.csv` or chardet) and convert before importing.
3. If converting in code, read the file, force_encoding to the known source encoding, then encode('UTF-8').

Example fix

// before
CsvImporter.new(File.read('outcomes.csv')) # raw Windows-1252 bytes
// after
raw = File.read('outcomes.csv', encoding: 'Windows-1252')
utf8 = raw.encode('UTF-8')
CsvImporter.new(utf8)

def utf8?(str)
  s = str&.force_encoding("utf-8")
  (s || "").valid_encoding?
end
raise "file is not UTF-8" unless utf8?(File.read(path))

def valid_utf8?(str)
  str.is_a?(String) && str.dup.force_encoding("UTF-8").valid_encoding?
end

begin
  importer.parse_file
rescue Outcomes::CsvImporter::ParseError => e
  logger.warn("CSV encoding problem: #{e.message}")
  # re-encode the file and retry once
end

When it happens

Trigger: Passing a CSV file whose bytes are not valid UTF-8 (e.g. saved as ISO-8859-1/Windows-1252 by Excel) into the outcome CSV import; embedding a smart quote or accented character in a non-UTF-8 codepage; reading the file with binary/ASCII-8BIT encoding and handing it to check_encoding directly.

Common situations: Spreadsheets exported from Excel on Windows in 'CSV (MS-DOS)' or ANSI format; files edited in editors using legacy encodings; downloads from legacy systems using ISO-8859-1.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• Database error ( )
• Empty file
• File has no data
• File has no outcomes data
• Friendly description is too long (maximum is 255 characters)

AI-assisted analysis of instructure/canvas-lms@1c9f0bb801 (2026-09-15). Data as JSON: /api/errors/3465f1cf450be3e7. Report an issue: GitHub.

Appendix: source

Thrown at lib/outcomes/csv_importer.rb:132

    def test_header_i18n
      header = @file.readline
      has_bom = header.start_with?((+"\xEF\xBB\xBF").force_encoding("ASCII-8BIT"))
      @file.rewind
      @file.read(3) if has_bom
      (header.count(";") > header.count(",")) ? ";" : ","
    end

    def file_line_count
      count = @file.each.inject(0) { |c, _line| c + 1 }
      @file.rewind
      count
    end

    def check_encoding(str)
      encoded = str&.force_encoding("utf-8")
      valid = (encoded || "").valid_encoding?
      raise ParseError, I18n.t("Not a valid utf-8 string: %{string}", string: str.inspect) unless valid

      encoded
    end

    def validate_headers(row, _index)
      main_columns_end = row.find_index("ratings") || row.length
      headers = row.slice(0, main_columns_end).map(&:to_sym)

      after_ratings = row[(main_columns_end + 1)..] || []
      after_ratings = after_ratings.compact_blank.map(&:to_s)
      raise ParseError, I18n.t("Invalid fields after ratings: %{fields}", fields: after_ratings.inspect) unless after_ratings.empty?

      missing = (REQUIRED_FIELDS - headers).map(&:to_s)
      raise ParseError, I18n.t("Missing required fields: %{fields}", fields: missing.inspect) unless missing.empty?

      invalid = (headers - OPTIONAL_FIELDS - REQUIRED_FIELDS).map(&:to_s)
      raise ParseError, I18n.t("Invalid fields: %{fields}", fields: invalid.inspect) unless invalid.empty?

View on GitHub (pinned to 1c9f0bb801)