{"record":{"id":"7e28c18da09a6548","repo":"gleam-lang/gleam","slug":"non-utf-8-path","errorCode":null,"errorMessage":"Non Utf-8 Path","messagePattern":"Non Utf-8 Path","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"compiler-cli/src/fs.rs","lineNumber":475,"sourceCode":"/// Walks through all Gleam module files in the directory, even if ignored,\n/// except for those in the `build/` directory. Excludes any Gleam files within\n/// invalid module paths, for example if they or a folder they're in contain a\n/// dot or a hyphen within their names.\npub fn gleam_files(dir: &Utf8Path) -> impl Iterator<Item = Utf8PathBuf> + '_ {\n    ignore::WalkBuilder::new(dir)\n        .follow_links(true)\n        .standard_filters(false)\n        .filter_entry(|entry| !is_gleam_build_dir(entry))\n        .build()\n        .filter_map(Result::ok)\n        .filter(|entry| {\n            entry\n                .file_type()\n                .map(|type_| type_.is_file())\n                .unwrap_or(false)\n        })\n        .map(ignore::DirEntry::into_path)\n        .map(|path| Utf8PathBuf::from_path_buf(path).expect(\"Non Utf-8 Path\"))\n        .filter(move |d| is_gleam_path(d, dir))\n}\n\n/// Walks through all native files in the directory, such as `.mjs` and `.erl`,\n/// even if ignored.\npub fn native_files(dir: &Utf8Path) -> impl Iterator<Item = Utf8PathBuf> + '_ {\n    ignore::WalkBuilder::new(dir)\n        .follow_links(true)\n        .standard_filters(false)\n        .filter_entry(|entry| !is_gleam_build_dir(entry))\n        .build()\n        .filter_map(Result::ok)\n        .filter(|entry| {\n            entry\n                .file_type()\n                .map(|type_| type_.is_file())\n                .unwrap_or(false)\n        })","sourceCodeStart":457,"sourceCodeEnd":493,"githubUrl":"https://github.com/gleam-lang/gleam/blob/7e623aa83da3776faee50ca4ab9a6c40124acd95/compiler-cli/src/fs.rs#L457-L493","documentation":"When enumerating Gleam source files, compiler-cli walks the directory tree with `ignore::WalkBuilder` (`follow_links(true)`, `standard_filters(false)`, build dir pruned) and converts every visited file to a `camino::Utf8PathBuf` via `Utf8PathBuf::from_path_buf(path).expect(\"Non Utf-8 Path\")` (compiler-cli/src/fs.rs:475). On Unix a filename is raw bytes; any file whose name is not valid UTF-8 makes `from_path_buf` return None and the walk panics, aborting the build.","triggerScenarios":"Any regular file with invalid-UTF-8 bytes in its name inside the walked source tree (src/, test/, or the directory given to the walker): files extracted from archives created with latin-1/Shift-JIS/HFS+ names, mangled fixtures or blobs, or names written by misbehaving tooling. Because `standard_filters(false)` disables ignore rules, even .gitignored files are visited and can trigger the panic.","commonSituations":"A stray fixture like `test\\xf1data` in src/ or test/; a teammate on another OS/encoding committing oddly named files (git stores filename bytes verbatim); vendor directories unpacked from third-party archives; junk files left by crashed editors or downloaders.","solutions":["Locate offending names with a byte-walking scan (see the validation snippet in the defense section)","Rename the file to an ASCII/UTF-8 name using a printf-built path (e.g. `mv \"$(printf 'src/test\\xf1data')\" src/test_f1data`) or delete it if it is junk","Re-run `gleam build` — the walk reruns each build, so the fix applies immediately","Re-extract the originating archive with the correct filename encoding (e.g. `unzip -O` variants or `convmv`) so the bad names do not come back"],"exampleFix":"# find files whose names are not valid UTF-8\npython3 - <<'PY'\nimport os\nfor root, dirs, files in os.walk(b\".\"):\n    if b\"/build\" in root:\n        continue\n    for n in dirs + files:\n        try:\n            n.decode(\"utf-8\")\n        except UnicodeDecodeError:\n            print(os.path.join(root, n))\nPY\n\n# rename the offender, then rebuild\nmv \"$(printf 'src/test\\xf1data')\" src/test_f1data && gleam build","handlingStrategy":"validation","validationCode":"# pre-build check: list any filename that is not valid UTF-8\npython3 - <<'PY'\nimport os, sys\nbad = []\nfor root, dirs, files in os.walk(b\".\"):\n    if b\"/build\" in root or b\"/.git\" in root:\n        continue\n    for n in dirs + files:\n        try:\n            n.decode(\"utf-8\")\n        except UnicodeDecodeError:\n            bad.append(os.path.join(root, n))\nprint(\"\\n\".join(map(repr, bad)) or \"all filenames UTF-8 safe\")\nsys.exit(1 if bad else 0)\nPY\ngleam build","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep repository filenames ASCII/UTF-8 only; add a CI filename lint","Remember .gitignore does NOT protect you: these walkers traverse ignored files too","Extract third-party archives with the correct filename charset (`convmv`, `unzip -O` variants)"],"tags":["gleam","filesystem","utf-8","filename","build","camino"],"backgroundTag":"non-utf8-filename","analyzedSha":"7e623aa83da3776faee50ca4ab9a6c40124acd95","analyzedAt":"2026-08-17T00:07:02.091Z","schemaVersion":2},"datasetVersion":"2026-08-17T04:17:16.089Z"}