{"record":{"id":"8d0e4d0a72806f35","repo":"astral-sh/ruff","slug":"truth-does-not-exist-ty-s-completion-evaluation","errorCode":null,"errorMessage":"{truth} does not exist: ty's completion evaluation must be run from the root of the ruff repository","messagePattern":"(.+?) does not exist: ty's completion evaluation must be run from the root of the ruff repository","errorType":"console","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/ty_completion_eval/src/main.rs","lineNumber":142,"sourceCode":"}\n\nfn main() -> anyhow::Result<ExitCode> {\n    let args = Cli::parse();\n\n    // The base path to which all CLI arguments are relative to.\n    let cwd = {\n        let cwd = std::env::current_dir().context(\"Failed to get the current working directory\")?;\n        SystemPathBuf::from_path_buf(cwd).map_err(|path| {\n            anyhow!(\n                \"The current working directory `{}` contains non-Unicode characters. \\\n                 ty only supports Unicode paths.\",\n                path.display()\n            )\n        })?\n    };\n    // Where we store our truth data.\n    let truth = cwd.join(\"crates\").join(\"ty_completion_eval\").join(\"truth\");\n    anyhow::ensure!(\n        truth.as_std_path().exists(),\n        \"{truth} does not exist: ty's completion evaluation must be run from the root \\\n         of the ruff repository\",\n        truth = truth.as_std_path().display(),\n    );\n\n    // The temporary directory at which we copy our truth\n    // data to. We do this because we can't use the truth\n    // data as-is with its `<CURSOR>` annotations (and perhaps\n    // any other future annotations we add).\n    let mut tmp_eval_dir = tempfile::Builder::new()\n        .prefix(\"ty-completion-eval-\")\n        .tempdir()\n        .context(\"Failed to create temporary directory\")?;\n    let tmp_eval_path = SystemPath::from_std_path(tmp_eval_dir.path())\n        .ok_or_else(|| {\n            anyhow::anyhow!(\n                \"Temporary directory path is not valid UTF-8: {}\",","sourceCodeStart":124,"sourceCodeEnd":160,"githubUrl":"https://github.com/astral-sh/ruff/blob/672bb4edf04c84f8b0753346359daee4057158f2/crates/ty_completion_eval/src/main.rs#L124-L160","documentation":"ty_completion_eval hard-codes its layout relative to the working directory: it joins `crates/ty_completion_eval/truth` onto the cwd and asserts the directory exists. The harness is tied to the ruff repository structure, so invoking it from any other directory fails this check before any work starts.","triggerScenarios":"Running `cargo run -p ty_completion_eval` from a subdirectory of the repo or from outside the checkout (cargo passes the invoking shell's cwd to the child process); a sparse/partial checkout that excluded the truth fixtures.","commonSituations":"Running the compiled binary directly from `$HOME` or `/tmp`; CI scripts that cd elsewhere before invoking; fresh clones with sparse-checkout rules missing `crates/ty_completion_eval/truth`.","solutions":["cd to the ruff repository root and re-run the exact command from the docs","Verify the fixtures exist: `test -d crates/ty_completion_eval/truth`","If using sparse checkout, reconfigure it to include `crates/ty_completion_eval`"],"exampleFix":"# before (run from a subdirectory)\ncrates/ $ cargo run -p ty_completion_eval -- all\n# after (run from repo root)\n$ cargo run -p ty_completion_eval -- all --threshold 0.4 --tasks crates/ty_completion_eval/completion-evaluation-tasks.csv","handlingStrategy":"validation","validationCode":"cd \"$(git rev-parse --show-toplevel 2>/dev/null)\" \\\n  && test -d crates/ty_completion_eval/truth \\\n  || { echo \"run ty_completion_eval from the ruff repo root\" >&2; exit 2; }","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always invoke the eval harness from the repository root","In scripts, `cd $(git rev-parse --show-toplevel)` first","Ensure sparse checkouts include crates/ty_completion_eval"],"tags":["evaluation","paths","developer-tools","repo-root"],"backgroundTag":null,"analyzedSha":"672bb4edf04c84f8b0753346359daee4057158f2","analyzedAt":"2026-08-16T08:54:05.464Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}