astral-sh/ruff · error · anyhow::Error

Detected project directory `{path}` contains non-Unicode cha

Error message

Detected project directory `{path}` contains non-Unicode characters. ty only supports Unicode paths.

What it means

ty_completion_bench locates the benchmark file's project by canonicalizing the path and walking ancestors for a `pyproject.toml`; when the found directory cannot convert to a `SystemPathBuf` because it contains non-UTF-8 bytes, ty's Unicode-only path layer rejects it with the offending path echoed.

Source

Thrown at crates/ty_completion_bench/src/main.rs:154

    offset: ruff_text_size::TextSize,
) -> anyhow::Result<Vec<Completion<'db>>> {
    let file = system_path_to_file(db, path)
        .with_context(|| format!("failed to get database file for `{path}`"))?;
    let settings = ty_ide::CompletionSettings::default();
    Ok(ty_ide::completion(
        db,
        &settings,
        CompletionCapabilities::default(),
        db.program_file(file),
        offset,
    ))
}

fn discover_project_directory(file: &SystemPath) -> anyhow::Result<SystemPathBuf> {
    for ancestor in file.as_std_path().canonicalize()?.ancestors() {
        if ancestor.join("pyproject.toml").exists() {
            return SystemPathBuf::from_path_buf(ancestor.to_path_buf()).map_err(|path| {
                anyhow!(
                    "Detected project directory `{path}` contains non-Unicode characters. \
                     ty only supports Unicode paths.",
                    path = path.display()
                )
            });
        }
    }
    anyhow::bail!("could not find `pyproject.toml` in any ancestor of `{file}`")
}

View on GitHub (pinned to 672bb4edf0)

Solutions

1. Move or rename the offending directory component so the whole path is valid UTF-8
2. Re-clone the benchmark project into a clean Unicode path and benchmark files there
3. Set a UTF-8 locale (LANG/LC_ALL) before creating directories so names come out UTF-8

Example fix

# before
cd $'proj\xff/bench' && cargo run -p ty_completion_bench -- snippet.py
# after
mv $'proj\xff' proj-safe && cd proj-safe/bench && cargo run -p ty_completion_bench -- snippet.py

python3 -c 'import os,sys; os.path.dirname(os.path.realpath(sys.argv[1])).encode("utf-8")' "$FILE" \
  || { echo "project path is not valid UTF-8" >&2; exit 2; }

// Rust (for harness embedders)
fn project_dir_is_unicode(file: &std::path::Path) -> bool {
    file.canonicalize().map(|p| p.to_str().is_some()).unwrap_or(false)
}

When it happens

Trigger: Benchmarking a file whose project directory — after `canonicalize()` resolves symlinks — contains byte sequences that are not valid UTF-8 (legacy-locale directory names, mangled archive extractions).

Common situations: Unix machines running without a UTF-8 locale; projects restored from backups/archives with encoding damage; sibling directories with byte-only names on the path from root to the project.

Related errors

• The current working directory `{}` contains non-Unicode char
• The current working directory `{}` contains non-Unicode char
• `uv sync` failed to run with exit code `{code}`, stderr: {st
• Temporary directory path is not valid UTF-8: {}
• truth source directory `{path}` contains invalid UTF-8

AI-assisted analysis of astral-sh/ruff@672bb4edf0 (2026-08-16). Data as JSON: /api/errors/e52736cffb825859. Report an issue: GitHub.