{"record":{"id":"4ed75fbbf29b85fb","repo":"instructure/canvas-lms","slug":"some-of-the-fields-contain-null-character","errorCode":null,"errorMessage":"Some of the fields contain NULL character","messagePattern":"Some of the fields contain NULL character","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"lib/sis/csv/user_importer.rb","lineNumber":79,"sourceCode":"          declared_user_type: row[\"declared_user_type\"],\n          password: row[\"password\"],\n          canvas_password_notification: row[\"canvas_password_notification\"],\n          ssha_password: row[\"ssha_password\"],\n          integration_id: row[\"integration_id\"],\n          short_name: row[\"short_name\"],\n          full_name: row[\"full_name\"],\n          sortable_name: row[\"sortable_name\"],\n          home_account: row[\"home_account\"],\n          lineno: row[\"lineno\"],\n          csv:,\n          row:,\n          authentication_provider_id: row[\"authentication_provider_id\"]\n        )\n      end\n\n      def validate(row)\n        if row.fields.any? { |v| v.to_s.include?(\"\\x00\") }\n          raise ImportError, \"Some of the fields contain NULL character\"\n        end\n      end\n    end\n  end\nend\n","sourceCodeStart":61,"sourceCodeEnd":85,"githubUrl":"https://github.com/instructure/canvas-lms/blob/1c9f0bb8013ed69c4f2efe11fd483025469b7e6c/lib/sis/csv/user_importer.rb#L61-L85","documentation":"Canvas SIS CSV import validation rejects any row whose CSV field values contain a literal NUL byte (\"\\x00\"). UserImporter.validate checks every field of the parsed row and raises ImportError before any user record is processed, because NUL bytes can corrupt downstream string handling and database writes. It is a defensive data-hygiene check on the raw CSV content.","triggerScenarios":"A user CSV row (e.g. user_id, email, name columns) contains an embedded NUL byte, so `row.fields.any? { |v| v.to_s.include?(\"\\x00\") }` is true when validate(row) is called during import of lib/sis/csv/user_importer.rb.","commonSituations":"CSV files exported from databases or legacy systems that embed NUL characters; binary files mistakenly renamed to .csv; files produced by buggy ETL scripts writing fixed-width buffers with NUL padding; UTF-16-encoded files interpreted as UTF-8.","solutions":["Open the CSV in an editor/script and strip NUL bytes before import: `File.write(path, File.read(path).gsub(\"\\x00\", ''))`","Re-export the source file as clean UTF-8 text without binary padding","Check the producing system/export job for the bug that injects NUL padding","Sanitize programmatically at parse time by mapping fields through `v.to_s.delete(\"\\x00\")` upstream of the importer"],"exampleFix":"// before\nCSV.foreach(path) { |row| importer.process(row) }\n// after\nCSV.foreach(path) do |row|\n  row = row.map { |v| v.to_s.delete(\"\\x00\") }\n  importer.process(row)\nend","handlingStrategy":"validation","validationCode":"def clean_csv_row?(row)\n  row.fields.all? { |v| !v.to_s.include?(\"\\x00\") }\nend\nraise 'NUL byte in CSV' unless clean_csv_row?(row)","typeGuard":null,"tryCatchPattern":"begin\n  importer.validate(row)\nrescue SIS::Base::ImportError => e\n  logger.warn(\"Skipping row: #{e.message}\")\nend","preventionTips":["Strip NUL bytes when exporting/transforming CSV files","Re-encode files to clean UTF-8 before SIS import","Validate input files with a linter/script before every import run"],"tags":["ruby","sis-import","csv","data-validation","null-byte"],"backgroundTag":"invalid-argument-format","analyzedSha":"1c9f0bb8013ed69c4f2efe11fd483025469b7e6c","analyzedAt":"2026-09-15T20:33:18.891Z","contentChangedAt":"2026-09-15T20:33:18.891Z","schemaVersion":2},"datasetVersion":"2026-09-23T02:17:17.105Z"}