{"record":{"id":"1772bc786c09603d","repo":"JuliusBrussee/caveman","slug":"system-runner-is-required","errorCode":null,"errorMessage":"system runner is required","messagePattern":"system runner is required","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"engine/evals/harness.go","lineNumber":199,"sourceCode":"func RunManifest(m Manifest, read func(name string) ([]byte, error)) (Report, error) {\n\treturn runManifest(context.Background(), m, read, nil)\n}\n\n// RunManifestWithQuality is RunManifest plus the LLM-in-the-loop quality layer.\n// The runner is injected so CI can use a deterministic local runner while release\n// jobs can call a real gateway/model.\nfunc RunManifestWithQuality(ctx context.Context, m Manifest, read func(name string) ([]byte, error), opts QualityOptions) (Report, error) {\n\tif opts.Runner == nil {\n\t\treturn Report{}, fmt.Errorf(\"quality runner is required\")\n\t}\n\treturn runManifest(ctx, m, read, &opts)\n}\n\n// RunManifestWithSystem runs a caller-supplied system-under-test through the\n// same graders and optional quality layer as the built-in engine.\nfunc RunManifestWithSystem(ctx context.Context, m Manifest, read func(name string) ([]byte, error), system TransformRunner, opts *QualityOptions) (Report, error) {\n\tif system == nil {\n\t\treturn Report{}, fmt.Errorf(\"system runner is required\")\n\t}\n\tif opts != nil && opts.Runner == nil {\n\t\treturn Report{}, fmt.Errorf(\"quality runner is required\")\n\t}\n\treturn runManifestWithSystem(ctx, m, read, system, opts)\n}\n\nfunc runManifest(ctx context.Context, m Manifest, read func(name string) ([]byte, error), quality *QualityOptions) (Report, error) {\n\tstore, err := ccr.OpenMemory()\n\tif err != nil {\n\t\treturn Report{}, fmt.Errorf(\"open recovery store: %w\", err)\n\t}\n\tdefer store.Close()\n\teng := engine.New(store, nil)\n\treturn runManifestWithSystem(ctx, m, read, engineSystem{eng: eng}, quality)\n}\n\nfunc runManifestWithSystem(ctx context.Context, m Manifest, read func(name string) ([]byte, error), system TransformRunner, quality *QualityOptions) (Report, error) {","sourceCodeStart":181,"sourceCodeEnd":217,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/engine/evals/harness.go#L181-L217","documentation":"Error \"system runner is required\" thrown in JuliusBrussee/caveman.","triggerScenarios":"Thrown at engine/evals/harness.go:199 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Provide a system runner to the eval harness."],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}