JuliusBrussee/caveman · error

resolve fixture directory: %w

Error message

resolve fixture directory: %w

What it means

RunDir failed at filepath.EvalSymlinks(root): the fixture directory path does not exist, or a symlink in the path is broken/loops. Resolving the real path up front is how the harness confines fixture reads to the root, so an unresolvable root aborts before any fixture is opened.

Source

Thrown at engine/evals/harness.go:117

	if err != nil {
		return Report{}, fmt.Errorf("read manifest: %w", err)
	}
	if err := yaml.Unmarshal(raw, &m); err != nil {
		return Report{}, fmt.Errorf("parse manifest: %w", err)
	}
	read := func(name string) ([]byte, error) {
		return fixturesFS.ReadFile("fixtures/" + name)
	}
	return RunManifest(m, read)
}

// RunDir replays a caller-supplied fixture directory containing manifest.yaml.
// Manifest file paths are slash-separated, relative, and confined to root even
// through symlinks; a custom empty corpus is a configuration error, never green.
func RunDir(root string) (Report, error) {
	resolvedRoot, err := filepath.EvalSymlinks(root)
	if err != nil {
		return Report{}, fmt.Errorf("resolve fixture directory: %w", err)
	}
	manifestRaw, err := os.ReadFile(filepath.Join(resolvedRoot, "manifest.yaml"))
	if err != nil {
		return Report{}, fmt.Errorf("read manifest: %w", err)
	}
	var m Manifest
	if err := yaml.Unmarshal(manifestRaw, &m); err != nil {
		return Report{}, fmt.Errorf("parse manifest: %w", err)
	}
	if len(m.Fixtures) == 0 {
		return Report{}, fmt.Errorf("manifest has no fixtures")
	}
	read := func(name string) ([]byte, error) {
		if !fs.ValidPath(name) || name == "." {
			return nil, fmt.Errorf("invalid fixture path %q", name)
		}
		candidate, err := filepath.EvalSymlinks(filepath.Join(resolvedRoot, filepath.FromSlash(name)))
		if err != nil {

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Check the path exists and the process can traverse it (ls, stat).
2. Use an absolute path or resolve relative to a known anchor (e.g. the test's working directory).
3. Fix or remove broken symlinks along the path.
4. If the directory moved after a repo restructure, update the --fixtures argument.

Example fix

# before
evals run --fixtures ./fixtures

# after: absolute path
evals run --fixtures "$(pwd)/fixtures"

func fixtureRootExists(root string) bool {
    resolved, err := filepath.EvalSymlinks(root)
    return err == nil
}

When it happens

Trigger: Calling evals.RunDir(root) with a typo'd path, a directory that was deleted, or a path containing a dangling/cyclic symlink.

Common situations: Passing a relative path from the wrong working directory; CI checkout layout changes so the fixtures dir moved; symlinked workspace (ln -s) pointing at a removed target.

Related errors

• caveman agent: dev ${kind} symlink escapes project root
• cave_tool_sandbox_entry_escapes_root
• cave_tool_sandbox_source_escapes_root
• caveman build: source graph symlink escapes project root
• File not found: {filepath}

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/2fba74ed32484a42. Report an issue: GitHub.