astral-sh/ruff · error

truth source directory

Error message

truth source directory `{path}` contains invalid UTF-8

What it means

When TaskSource::all enumerates truth directories, each entry's path is converted with SystemPath::from_std_path, which only succeeds for valid UTF-8 paths. This error is thrown when a truth source directory's filesystem path contains invalid UTF-8 bytes, since the tool stores paths as Rust strings.

Solutions

1. Rename the offending truth directory to a valid UTF-8 name
2. Find it with `find crates/ty_completion_eval/truth -maxdepth 1 | grep -Pv '^[\x00-\x7F]*$'` or `ls -b` and delete/recreate it
3. Extract archives with UTF-8-safe filename conversion before placing fixtures in truth/

Example fix

// before (shell)
mv $'truth/bad\xffname' truth/badname
// after (shell)
ls -b truth/ && mv truth/'bad\xffname' truth/badname  # verify UTF-8 name

fn validate_truth_dir(root: &std::path::Path) -> std::io::Result<()> {
    for entry in std::fs::read_dir(root)? {
        let entry = entry?;
        if entry.path().is_dir() && entry.path().to_str().is_none() {
            return Err(std::io::Error::new(
                std::io::ErrorKind::InvalidData,
                format!("non-UTF-8 directory name: {}", entry.path().display()),
            ));
        }
    }
    Ok(())
}

fn is_utf8_path(p: &std::path::Path) -> bool {
    p.to_str().is_some()
}

match TaskSource::all() {
    Ok(sources) => sources,
    Err(e) if e.to_string().contains("invalid UTF-8") => {
        eprintln!("fix the non-UTF-8 directory name under the truth root: {e}");
        std::process::exit(2);
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: A directory under the truth folder has a non-UTF-8 name (created on Linux with raw bytes, e.g. from a bad filename or shell escape), and TaskSource::all iterates it.

Common situations: A contributor creates a fixture directory with a mangled/non-ASCII-bytes name on a non-UTF-8 locale filesystem; extraction of an archive produced byte-invalid filenames inside the truth directory.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• path ` ` is not valid UTF-8
• Expected a UTF-8 working directory
• could not find any ` ` directives in any of the files in
• project directory ` ` does not contain a base name
• System should be writable

AI-assisted analysis of astral-sh/ruff@5bb508f594 (2026-09-05). Data as JSON: /api/errors/2623bd27b5549a6e. Report an issue: GitHub.

Appendix: source

Thrown at crates/ty_completion_eval/src/main.rs:416

}

impl TaskSource {
    fn all(src_dir: &SystemPath) -> anyhow::Result<Vec<TaskSource>> {
        let mut sources = vec![];
        let read_dir = src_dir
            .as_std_path()
            .read_dir()
            .with_context(|| format!("failed to read directory entries in `{src_dir}`"))?;
        for result in read_dir {
            let dent = result
                .with_context(|| format!("failed to get directory entry from `{src_dir}`"))?;
            let path = dent.path();
            if !path.is_dir() {
                continue;
            }

            let dir = SystemPath::from_std_path(&path).ok_or_else(|| {
                anyhow::anyhow!(
                    "truth source directory `{path}` contains invalid UTF-8",
                    path = path.display()
                )
            })?;
            sources.push(TaskSource::new(dir)?);
        }
        // Sort our sources so that we always run in the same order.
        // And also so that the CSV output is deterministic across
        // all platforms.
        sources.sort_by(|source1, source2| source1.name.cmp(&source2.name));
        Ok(sources)
    }

    fn new(dir: &SystemPath) -> anyhow::Result<TaskSource> {
        let name = dir.file_name().ok_or_else(|| {
            anyhow::anyhow!("truth source directory `{dir}` does not contain a base name")
        })?;

View on GitHub (pinned to 5bb508f594)