{"record":{"id":"065e22f50d30c4d4","repo":"JuliusBrussee/caveman","slug":"read-manifest-w","errorCode":null,"errorMessage":"read manifest: %w","messagePattern":"read manifest: %w","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"engine/evals/harness.go","lineNumber":100,"sourceCode":"\tPassed       bool            `json:\"passed\"`\n\tTokensBefore int             `json:\"tokens_before\"`\n\tTokensAfter  int             `json:\"tokens_after\"`\n\tTokenRatio   float64         `json:\"token_ratio\"`\n\tWordsBefore  int             `json:\"words_before\"`\n\tWordsAfter   int             `json:\"words_after\"`\n\tWordRatio    float64         `json:\"word_ratio\"`\n\tProbes       ProbeSummary    `json:\"probes\"`\n\tQuality      *QualitySummary `json:\"quality,omitempty\"`\n\tFixtures     []FixtureReport `json:\"fixtures\"`\n}\n\n// Run replays the embedded fixture set behind its graders and returns the\n// report. It runs the engine's configured quality gate.\nfunc Run() (Report, error) {\n\tvar m Manifest\n\traw, err := fixturesFS.ReadFile(\"fixtures/manifest.yaml\")\n\tif err != nil {\n\t\treturn Report{}, fmt.Errorf(\"read manifest: %w\", err)\n\t}\n\tif err := yaml.Unmarshal(raw, &m); err != nil {\n\t\treturn Report{}, fmt.Errorf(\"parse manifest: %w\", err)\n\t}\n\tread := func(name string) ([]byte, error) {\n\t\treturn fixturesFS.ReadFile(\"fixtures/\" + name)\n\t}\n\treturn RunManifest(m, read)\n}\n\n// RunDir replays a caller-supplied fixture directory containing manifest.yaml.\n// Manifest file paths are slash-separated, relative, and confined to root even\n// through symlinks; a custom empty corpus is a configuration error, never green.\nfunc RunDir(root string) (Report, error) {\n\tresolvedRoot, err := filepath.EvalSymlinks(root)\n\tif err != nil {\n\t\treturn Report{}, fmt.Errorf(\"resolve fixture directory: %w\", err)\n\t}","sourceCodeStart":82,"sourceCodeEnd":118,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/engine/evals/harness.go#L82-L118","documentation":"evals.Run() could not read the embedded fixture manifest at fixtures/manifest.yaml from the compiled-in filesystem. Because the fixtures are embedded at build time, this error almost always means the binary was built from a tree where the embed directory was missing or the embed directive drifted.","triggerScenarios":"Calling evals.Run() (or the evals run CLI subcommand) in a build where the fixtures embed FS has no fixtures/manifest.yaml — wrong build tags, moved fixture directory, or a stale/partial build cache.","commonSituations":"Building after moving engine/evals/fixtures; go build cache serving a stale embed; vendoring/forking that dropped the fixtures directory; empty-directory tricks used to satisfy embed failing for the manifest specifically.","solutions":["Verify engine/evals/fixtures/manifest.yaml exists in the source tree you built from.","Force a clean rebuild (go clean -cache; go build) to rule out stale embedded data.","If you forked the repo, restore the fixtures directory or point RunDir at an external fixture root instead.","Check the embed directive still matches the fixtures path."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// At build-check time, confirm the embedded fixtures exist\nif _, err := fixturesFS.ReadFile(\"fixtures/manifest.yaml\"); err != nil {\n    log.Fatal(\"embedded fixtures missing — rebuild from a complete source tree\")\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Run a smoke eval after every build in CI.","Keep engine/evals/fixtures under version control with the manifest committed.","go clean -cache when embed contents seem stale."],"tags":["evals","embed","build","manifest"],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}