astral-sh/ruff · error

path ` ` is not valid UTF-8

Error message

path `{}` is not valid UTF-8

What it means

copy_project copies a truth fixture project and enumerates its entries; SystemPath::from_std_path rejects non-UTF-8 paths, so when a file or subdirectory inside the fixture has a byte-invalid name this anyhow error is returned, since the eval tool only handles UTF-8 paths.

Solutions

1. Rename or delete the non-UTF-8 entry inside the fixture project directory
2. Locate it with `find <fixture-dir> | grep -P '[^\x00-\x7F]'` or `ls -b <fixture-dir>`
3. Regenerate the fixture from a clean source rather than copying extracted archives

Example fix

// before
$ cp -a ~/Downloads/project_fixture crates/ty_completion_eval/truth/  # contains bad\xff.py
// after
$ find crates/ty_completion_eval/truth/project_fixture -name $'\xff*' -delete
$ uv run --package ty_completion_eval -- all --threshold 0.4 ...

fn validate_fixture_utf8(dir: &std::path::Path) -> std::io::Result<()> {
    for entry in walkdir(dir) {
        if entry.to_str().is_none() {
            return Err(std::io::Error::new(
                std::io::ErrorKind::InvalidData,
                format!("non-UTF-8 path in fixture: {}", entry.display()),
            ));
        }
    }
    Ok(())
}

fn all_utf8(dir: &std::path::Path) -> bool {
    std::fs::read_dir(dir).map(|rd| rd.filter_map(|e| e.ok()).all(|e| e.path().to_str().is_some())).unwrap_or(false)
}

match to_tasks(...) {
    Ok(tasks) => tasks,
    Err(e) if e.to_string().contains("not valid UTF-8") => {
        eprintln!("rename the non-UTF-8 entry in the fixture: {e}");
        std::process::exit(2);
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: Running an evaluation task whose project fixture directory contains a file or directory with non-UTF-8 bytes in its name; hit during to_tasks -> copy_project while iterating directory entries.

Common situations: Archive extraction, symlink weirdness, or a Python/shell script created a file with escaped byte names inside the fixture; copied fixture from a filesystem with non-UTF-8 encoding.

Related errors

• truth source directory
• Expected a UTF-8 working directory
• could not find any ` ` directives in any of the files in
• project directory ` ` does not contain a base name
• System should be writable

AI-assisted analysis of astral-sh/ruff@5bb508f594 (2026-09-05). Data as JSON: /api/errors/02b142eef365a479. Report an issue: GitHub.

Appendix: source

Thrown at crates/ty_completion_eval/src/main.rs:561

fn copy_project(src_dir: &SystemPath, dst_dir: &SystemPath) -> anyhow::Result<Vec<Cursor>> {
    std::fs::create_dir_all(dst_dir).with_context(|| dst_dir.to_string())?;

    let mut cursors = vec![];
    let it = walkdir::WalkDir::new(src_dir.as_std_path())
        .sort_by_file_name()
        .into_iter()
        .filter_entry(|dent| {
            !dent
                .file_name()
                .to_str()
                .is_some_and(|name| name.starts_with('.'))
        });
    for result in it {
        let dent =
            result.with_context(|| format!("failed to get directory entry from {src_dir}"))?;

        let src = SystemPath::from_std_path(dent.path()).ok_or_else(|| {
            anyhow::anyhow!("path `{}` is not valid UTF-8", dent.path().display())
        })?;
        let name = src
            .strip_prefix(src_dir)
            .expect("descendent of `src_dir` must start with `src`");
        // let name = src
        // .file_name()
        // .ok_or_else(|| anyhow::anyhow!("path `{src}` is missing a basename"))?;
        let dst = dst_dir.join(name);
        if dent.file_type().is_dir() {
            std::fs::create_dir_all(dst.as_std_path())
                .with_context(|| format!("failed to create directory `{dst}`"))?;
        } else {
            cursors.extend(copy_file(src, &dst)?);
        }
    }
    anyhow::ensure!(
        !cursors.is_empty(),
        "could not find any `<CURSOR>` directives in any of the files in `{src_dir}`",

View on GitHub (pinned to 5bb508f594)