{"record":{"id":"1777cfda47384f8d","repo":"can1357/oh-my-pi","slug":"terminal-bench-dataset-source-is-not-a-directory","errorCode":null,"errorMessage":"Terminal-Bench dataset source is not a directory: ${source}","messagePattern":"Terminal-Bench dataset source is not a directory: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"packages/metaharness/src/tb/dataset.ts","lineNumber":55,"sourceCode":"\tconst basename = trimmed.slice(trimmed.lastIndexOf(\"/\") + 1).replace(/\\.git$/, \"\");\n\tif (!basename) throw new Error(`Cannot determine repository name from dataset source: ${source}`);\n\treturn basename;\n}\n\nfunction gitError(action: string, source: string, stderr: Uint8Array): Error {\n\tconst detail = new TextDecoder().decode(stderr).trim();\n\treturn new Error(`${action} ${source} failed${detail ? `: ${detail}` : \"\"}`);\n}\n\nexport async function resolveDataset(source: string, cacheDir: string): Promise<string> {\n\tconst sourceKind = await directoryStat(source);\n\tif (sourceKind === \"directory\") {\n\t\tconst absoluteSource = path.resolve(source);\n\t\tconst nestedTasks = path.join(absoluteSource, \"tasks\");\n\t\tconst tasksDir = (await directoryStat(nestedTasks)) === \"directory\" ? nestedTasks : absoluteSource;\n\t\treturn validateTasksDir(tasksDir);\n\t}\n\tif (sourceKind === \"other\") throw new Error(`Terminal-Bench dataset source is not a directory: ${source}`);\n\n\tconst isGitUrl = source.startsWith(\"http://\") || source.startsWith(\"https://\") || source.startsWith(\"git@\");\n\tif (!isGitUrl) throw new Error(`Terminal-Bench dataset source does not exist: ${source}`);\n\n\tconst absoluteCache = path.resolve(cacheDir);\n\tawait fs.mkdir(absoluteCache, { recursive: true });\n\tconst checkoutDir = path.join(absoluteCache, repositoryName(source));\n\tconst gitDir = path.join(checkoutDir, \".git\");\n\tif ((await directoryStat(gitDir)) === \"directory\") {\n\t\tconst result = await $`git pull --ff-only`.cwd(checkoutDir).quiet().nothrow();\n\t\tif (result.exitCode !== 0) throw gitError(\"Updating\", source, result.stderr);\n\t} else {\n\t\tconst result = await $`git clone --depth 1 ${source} ${checkoutDir}`.quiet().nothrow();\n\t\tif (result.exitCode !== 0) throw gitError(\"Cloning\", source, result.stderr);\n\t}\n\n\treturn validateTasksDir(path.join(checkoutDir, \"tasks\"));\n}","sourceCodeStart":37,"sourceCodeEnd":73,"githubUrl":"https://github.com/can1357/oh-my-pi/blob/969062200754ea02cfac922e5ebb8c608c079e15/packages/metaharness/src/tb/dataset.ts#L37-L73","documentation":"resolveDataset classifies the source via directoryStat: 'directory', 'other' (exists but not a directory), or missing. When the source exists but is a file/symlink-to-file/special file, it cannot be used as a dataset, so this error is thrown.","triggerScenarios":"Calling resolveDataset (from main/tasksDir) with a --dataset value that resolves to a regular file (e.g. an archive, a task.toml itself, or a tarball path).","commonSituations":"Passing a zip/tar of the dataset instead of an extracted directory; passing a single task's task.toml; shell glob expanding to a file.","solutions":["Point --dataset at a directory (or its tasks/ subdir) containing per-task task.toml files","Extract the archive first and pass the extracted directory","Fix shell globs so they expand to the dataset directory"],"exampleFix":"// before\ntb run --dataset tb-tasks.tar.gz\n// after\ntar xzf tb-tasks.tar.gz\ntb run --dataset ./tb-tasks","handlingStrategy":"validation","validationCode":"import { stat } from \"node:fs/promises\";\nconst s = await stat(source);\nif (!s.isDirectory()) throw new Error(`${source} must be a directory (extract archives first)`);","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Extract archives before passing as --dataset","Don't pass single files (task.toml, tarballs) as dataset sources","Quote globs to avoid unintended file expansion"],"tags":["filesystem","dataset","wrong-type"],"backgroundTag":"path-is-not-a-directory","analyzedSha":"969062200754ea02cfac922e5ebb8c608c079e15","analyzedAt":"2026-08-31T10:29:35.737Z","schemaVersion":2},"datasetVersion":"2026-08-31T14:17:45.589Z"}