JuliusBrussee/caveman · error
fixture path escapes root: %q
Error message
fixture path escapes root: %q
What it means
A fixture path that passed basic validation resolved (through symlinks) to a location outside the fixture root. The harness EvalSymlinks each candidate and requires the result to stay under resolvedRoot; escaping via '../' traversal or a symlink pointing outward is a path-traversal attempt and is rejected. This is a security control, not a convenience check.
Source
Thrown at engine/evals/harness.go:140
}
var m Manifest
if err := yaml.Unmarshal(manifestRaw, &m); err != nil {
return Report{}, fmt.Errorf("parse manifest: %w", err)
}
if len(m.Fixtures) == 0 {
return Report{}, fmt.Errorf("manifest has no fixtures")
}
read := func(name string) ([]byte, error) {
if !fs.ValidPath(name) || name == "." {
return nil, fmt.Errorf("invalid fixture path %q", name)
}
candidate, err := filepath.EvalSymlinks(filepath.Join(resolvedRoot, filepath.FromSlash(name)))
if err != nil {
return nil, err
}
rel, err := filepath.Rel(resolvedRoot, candidate)
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return nil, fmt.Errorf("fixture path escapes root: %q", name)
}
return os.ReadFile(candidate)
}
return RunManifest(m, read)
}
// RunWithQuality replays the embedded fixture set and also sends baseline and
// compressed prompts through a model runner before grading the answers.
func RunWithQuality(ctx context.Context, opts QualityOptions) (Report, error) {
m, err := EmbeddedManifest()
if err != nil {
return Report{}, err
}
read := func(name string) ([]byte, error) {
return fixturesFS.ReadFile("fixtures/" + name)
}
return RunManifestWithQuality(ctx, m, read, opts)
}View on GitHub (pinned to 27d5a3981a)
Solutions
- Keep all fixture files physically (or symlinking to targets) inside the fixture root.
- Replace '../' traversal in manifest paths with paths relative to the fixture root.
- If fixtures must live elsewhere, pass that location as its own --fixtures root instead of symlinking out.
- Audit manifests from third parties before running them through this harness.
Example fix
# before - name: shared path: ../../shared_corpus/x.txt # after: run with the shared corpus as its own root caveman-engine evals run --fixtures /path/to/shared_corpus
Defensive patterns
Strategy: validation
Validate before calling
func fixtureInsideRoot(root, name string) bool {
candidate, err := filepath.EvalSymlinks(filepath.Join(root, filepath.FromSlash(name)))
if err != nil { return false }
rel, err := filepath.Rel(root, candidate)
return err == nil && rel != ".." && !strings.HasPrefix(rel, ".."+string(filepath.Separator))
} Prevention
- Never use ../ traversal in manifest fixture paths.
- Keep symlinks inside the fixture root pointing at targets also inside the root.
- Pass an out-of-tree corpus as its own --fixtures root instead of symlinking into one.
- Audit third-party manifests before running them.
When it happens
Trigger: A manifest entry like '../../etc/passwd', or a symlink inside the fixtures directory pointing to a file outside it, so filepath.Rel(resolvedRoot, candidate) starts with '..'.
Common situations: Symlinked fixture directories shared between projects; a malicious or corrupted manifest from an untrusted source; monorepo setups where fixtures symlink into a shared store outside the root.
Related errors
- caveman-code: path escapes the workspace: ${candidate}
- cave_live_eval_sandbox_profile_escapes_root
- caveman agent: file source escapes project root
- caveman agent: dev ${kind} symlink escapes project root
- cave_memory_tenant_invalid
AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15).
Data as JSON: /api/errors/de6332407e9420f1.
Report an issue: GitHub.