{"record":{"id":"2623bd27b5549a6e","repo":"astral-sh/ruff","slug":"truth-source-directory-path-contains-invalid-u","errorCode":null,"errorMessage":"truth source directory `{path}` contains invalid UTF-8","messagePattern":"truth source directory `(.+?)` contains invalid UTF-8","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/ty_completion_eval/src/main.rs","lineNumber":416,"sourceCode":"}\n\nimpl TaskSource {\n    fn all(src_dir: &SystemPath) -> anyhow::Result<Vec<TaskSource>> {\n        let mut sources = vec![];\n        let read_dir = src_dir\n            .as_std_path()\n            .read_dir()\n            .with_context(|| format!(\"failed to read directory entries in `{src_dir}`\"))?;\n        for result in read_dir {\n            let dent = result\n                .with_context(|| format!(\"failed to get directory entry from `{src_dir}`\"))?;\n            let path = dent.path();\n            if !path.is_dir() {\n                continue;\n            }\n\n            let dir = SystemPath::from_std_path(&path).ok_or_else(|| {\n                anyhow::anyhow!(\n                    \"truth source directory `{path}` contains invalid UTF-8\",\n                    path = path.display()\n                )\n            })?;\n            sources.push(TaskSource::new(dir)?);\n        }\n        // Sort our sources so that we always run in the same order.\n        // And also so that the CSV output is deterministic across\n        // all platforms.\n        sources.sort_by(|source1, source2| source1.name.cmp(&source2.name));\n        Ok(sources)\n    }\n\n    fn new(dir: &SystemPath) -> anyhow::Result<TaskSource> {\n        let name = dir.file_name().ok_or_else(|| {\n            anyhow::anyhow!(\"truth source directory `{dir}` does not contain a base name\")\n        })?;\n","sourceCodeStart":398,"sourceCodeEnd":434,"githubUrl":"https://github.com/astral-sh/ruff/blob/5bb508f59481c1a02d5c7ff8568b9ffd8a763eef/crates/ty_completion_eval/src/main.rs#L398-L434","documentation":"When TaskSource::all enumerates truth directories, each entry's path is converted with SystemPath::from_std_path, which only succeeds for valid UTF-8 paths. This error is thrown when a truth source directory's filesystem path contains invalid UTF-8 bytes, since the tool stores paths as Rust strings.","triggerScenarios":"A directory under the truth folder has a non-UTF-8 name (created on Linux with raw bytes, e.g. from a bad filename or shell escape), and TaskSource::all iterates it.","commonSituations":"A contributor creates a fixture directory with a mangled/non-ASCII-bytes name on a non-UTF-8 locale filesystem; extraction of an archive produced byte-invalid filenames inside the truth directory.","solutions":["Rename the offending truth directory to a valid UTF-8 name","Find it with `find crates/ty_completion_eval/truth -maxdepth 1 | grep -Pv '^[\\x00-\\x7F]*$'` or `ls -b` and delete/recreate it","Extract archives with UTF-8-safe filename conversion before placing fixtures in truth/"],"exampleFix":"// before (shell)\nmv $'truth/bad\\xffname' truth/badname\n// after (shell)\nls -b truth/ && mv truth/'bad\\xffname' truth/badname  # verify UTF-8 name","handlingStrategy":"validation","validationCode":"fn validate_truth_dir(root: &std::path::Path) -> std::io::Result<()> {\n    for entry in std::fs::read_dir(root)? {\n        let entry = entry?;\n        if entry.path().is_dir() && entry.path().to_str().is_none() {\n            return Err(std::io::Error::new(\n                std::io::ErrorKind::InvalidData,\n                format!(\"non-UTF-8 directory name: {}\", entry.path().display()),\n            ));\n        }\n    }\n    Ok(())\n}","typeGuard":"fn is_utf8_path(p: &std::path::Path) -> bool {\n    p.to_str().is_some()\n}","tryCatchPattern":"match TaskSource::all() {\n    Ok(sources) => sources,\n    Err(e) if e.to_string().contains(\"invalid UTF-8\") => {\n        eprintln!(\"fix the non-UTF-8 directory name under the truth root: {e}\");\n        std::process::exit(2);\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Keep fixture directory names ASCII or valid UTF-8 only","Extract archives with UTF-8 filename conversion enabled","Run `ls -b` over the truth root after bulk-copying fixtures"],"tags":["rust","utf-8","filesystem","completion-eval"],"backgroundTag":"invalid-utf8-path","analyzedSha":"5bb508f59481c1a02d5c7ff8568b9ffd8a763eef","analyzedAt":"2026-09-05T10:32:37.492Z","contentChangedAt":"2026-09-05T10:32:37.492Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}