{"record":{"id":"361ecfe5381815d5","repo":"gleam-lang/gleam","slug":"error-printing-produced-invalid-utf8","errorCode":null,"errorMessage":"Error printing produced invalid utf8","messagePattern":"Error printing produced invalid utf8","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"compiler-core/src/error.rs","lineNumber":1001,"sourceCode":"        .map(|(option, _)| format!(\"Did you mean `{option}`?\"))\n}\n\nfn to_ordinal(value: u32) -> String {\n    match value % 10 {\n        // All numbers starting with 1 end in `th` (11th, 12th, 13th, etc.)\n        _ if value / 10 == 1 => format!(\"{value}th\"),\n        1 => format!(\"{value}st\"),\n        2 => format!(\"{value}nd\"),\n        3 => format!(\"{value}rd\"),\n        _ => format!(\"{value}th\"),\n    }\n}\n\nimpl Error {\n    pub fn pretty_string(&self) -> String {\n        let mut nocolor = Buffer::no_color();\n        self.pretty(&mut nocolor);\n        String::from_utf8(nocolor.into_inner()).expect(\"Error printing produced invalid utf8\")\n    }\n\n    pub fn pretty(&self, buffer: &mut Buffer) {\n        for diagnostic in self.to_diagnostics() {\n            diagnostic.write(buffer);\n            writeln!(buffer).expect(\"write new line after diagnostic\");\n        }\n    }\n\n    pub fn skipped_files_diagnostics(&self) -> HashMap<Utf8PathBuf, Diagnostic> {\n        let Error::Type {\n            skipped_modules, ..\n        } = self\n        else {\n            return HashMap::new();\n        };\n\n        skipped_modules","sourceCodeStart":983,"sourceCodeEnd":1019,"githubUrl":"https://github.com/gleam-lang/gleam/blob/49f8762da595eb3c8f576656a4200a142ca188c9/compiler-core/src/error.rs#L983-L1019","documentation":"`Error::pretty_string` renders a compiler diagnostic into an in-memory `codespan` `Buffer` with colors disabled, then converts the bytes to a `String` with `String::from_utf8(...).expect(\"Error printing produced invalid utf8\")`. The library assumes its own diagnostic rendering always emits valid UTF-8 (snippets are carved from UTF-8 source files); if the rendered bytes are not valid UTF-8, this invariant is broken and the call panics.","triggerScenarios":"Calling `Error::pretty_string()` (directly or via the CLI's error reporting path) when the diagnostic text assembled from a source snippet or file content includes bytes that are not valid UTF-8 — e.g. reading a source file with invalid encoding on a platform that decodes paths/content lossily, or a bug where raw bytes from a non-UTF-8 file are spliced into the output buffer.","commonSituations":"Compiling a Gleam project on a machine where source files or paths contain non-UTF-8 bytes (legacy encodings, corrupted files) and then triggering a compiler error that quotes those bytes; historically also an issue with locale-dependent output handling.","solutions":["Ensure all project source files are saved as valid UTF-8; re-encode offending files (e.g. `iconv -f LATIN1 -t UTF-8 file.gleam > out && mv out file.gleam`)","Upgrade Gleam — the compiler increasingly validates/lossily-converts non-UTF-8 input before diagnostics are built","If embedding this crate, call `Error::pretty(&mut buffer)` yourself and handle the `Vec<u8>` with `String::from_utf8_lossy` instead of using `pretty_string`"],"exampleFix":"// before (library consumer)\nlet text = error.pretty_string();\n// after\nlet mut buf = Buffer::no_color();\nerror.pretty(&mut buf);\nlet text = String::from_utf8_lossy(buf.into_inner()).into_owned();","handlingStrategy":"validation","validationCode":"// ensure sources are valid UTF-8 before compiling:\nlet bytes = std::fs::read(path)?;\nif std::str::from_utf8(&bytes).is_err() {\n    eprintln!(\"{path} is not valid UTF-8; re-encode it before compiling\");\n}","typeGuard":"fn is_utf8_file(p: &std::path::Path) -> bool {\n    std::fs::read(p).map(|b| std::str::from_utf8(&b).is_ok()).unwrap_or(false)\n}","tryCatchPattern":"// avoid pretty_string; render to bytes and lossy-convert:\nlet mut buf = Buffer::no_color();\nerror.pretty(&mut buf);\nlet text = String::from_utf8_lossy(buf.into_inner());","preventionTips":["Save all Gleam sources as UTF-8 (configure editor encoding)","Re-encode legacy files with iconv before committing them","When embedding the crate, use String::from_utf8_lossy over expect-based pretty_string"],"tags":["rust","compiler","utf8","diagnostics"],"backgroundTag":"internal-invariant-violation","analyzedSha":"49f8762da595eb3c8f576656a4200a142ca188c9","analyzedAt":"2026-09-14T11:14:59.388Z","contentChangedAt":"2026-09-14T11:14:59.388Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}