{"record":{"id":"de6332407e9420f1","repo":"JuliusBrussee/caveman","slug":"fixture-path-escapes-root-q","errorCode":null,"errorMessage":"fixture path escapes root: %q","messagePattern":"fixture path escapes root: %q","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"engine/evals/harness.go","lineNumber":140,"sourceCode":"\t}\n\tvar m Manifest\n\tif err := yaml.Unmarshal(manifestRaw, &m); err != nil {\n\t\treturn Report{}, fmt.Errorf(\"parse manifest: %w\", err)\n\t}\n\tif len(m.Fixtures) == 0 {\n\t\treturn Report{}, fmt.Errorf(\"manifest has no fixtures\")\n\t}\n\tread := func(name string) ([]byte, error) {\n\t\tif !fs.ValidPath(name) || name == \".\" {\n\t\t\treturn nil, fmt.Errorf(\"invalid fixture path %q\", name)\n\t\t}\n\t\tcandidate, err := filepath.EvalSymlinks(filepath.Join(resolvedRoot, filepath.FromSlash(name)))\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\trel, err := filepath.Rel(resolvedRoot, candidate)\n\t\tif err != nil || rel == \"..\" || strings.HasPrefix(rel, \"..\"+string(filepath.Separator)) {\n\t\t\treturn nil, fmt.Errorf(\"fixture path escapes root: %q\", name)\n\t\t}\n\t\treturn os.ReadFile(candidate)\n\t}\n\treturn RunManifest(m, read)\n}\n\n// RunWithQuality replays the embedded fixture set and also sends baseline and\n// compressed prompts through a model runner before grading the answers.\nfunc RunWithQuality(ctx context.Context, opts QualityOptions) (Report, error) {\n\tm, err := EmbeddedManifest()\n\tif err != nil {\n\t\treturn Report{}, err\n\t}\n\tread := func(name string) ([]byte, error) {\n\t\treturn fixturesFS.ReadFile(\"fixtures/\" + name)\n\t}\n\treturn RunManifestWithQuality(ctx, m, read, opts)\n}","sourceCodeStart":122,"sourceCodeEnd":158,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/engine/evals/harness.go#L122-L158","documentation":"A fixture path that passed basic validation resolved (through symlinks) to a location outside the fixture root. The harness EvalSymlinks each candidate and requires the result to stay under resolvedRoot; escaping via '../' traversal or a symlink pointing outward is a path-traversal attempt and is rejected. This is a security control, not a convenience check.","triggerScenarios":"A manifest entry like '../../etc/passwd', or a symlink inside the fixtures directory pointing to a file outside it, so filepath.Rel(resolvedRoot, candidate) starts with '..'.","commonSituations":"Symlinked fixture directories shared between projects; a malicious or corrupted manifest from an untrusted source; monorepo setups where fixtures symlink into a shared store outside the root.","solutions":["Keep all fixture files physically (or symlinking to targets) inside the fixture root.","Replace '../' traversal in manifest paths with paths relative to the fixture root.","If fixtures must live elsewhere, pass that location as its own --fixtures root instead of symlinking out.","Audit manifests from third parties before running them through this harness."],"exampleFix":"# before\n- name: shared\n  path: ../../shared_corpus/x.txt\n\n# after: run with the shared corpus as its own root\ncaveman-engine evals run --fixtures /path/to/shared_corpus","handlingStrategy":"validation","validationCode":"func fixtureInsideRoot(root, name string) bool {\n    candidate, err := filepath.EvalSymlinks(filepath.Join(root, filepath.FromSlash(name)))\n    if err != nil { return false }\n    rel, err := filepath.Rel(root, candidate)\n    return err == nil && rel != \"..\" && !strings.HasPrefix(rel, \"..\"+string(filepath.Separator))\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never use ../ traversal in manifest fixture paths.","Keep symlinks inside the fixture root pointing at targets also inside the root.","Pass an out-of-tree corpus as its own --fixtures root instead of symlinking into one.","Audit third-party manifests before running them."],"tags":["security","path-traversal","evals","symlink"],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}