{"record":{"id":"9697af4d3985dc86","repo":"vxcontrol/pentagi","slug":"test-case-has-no-prompt-or-messages","errorCode":null,"errorMessage":"test case has no prompt or messages","messagePattern":"test case has no prompt or messages","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"backend/pkg/providers/tester/runner.go","lineNumber":301,"sourceCode":"\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\tcase len(req.testCase.Messages()) > 0:\n\t\t// messages without tools\n\t\tresponse, err = req.provider.CallEx(\n\t\t\tctx,\n\t\t\treq.agentType,\n\t\t\treq.testCase.Messages(),\n\t\t\treq.testCase.StreamingCallback(),\n\t\t)\n\tcase req.testCase.Prompt() != \"\":\n\t\t// simple prompt\n\t\tresponse, err = req.provider.Call(ctx, req.agentType, req.testCase.Prompt())\n\tdefault:\n\t\treturn testdata.TestResult{}, fmt.Errorf(\"test case has no prompt or messages\")\n\t}\n\n\tlatency := time.Since(startTime)\n\n\tif err != nil {\n\t\treturn testdata.TestResult{\n\t\t\tID:          req.testCase.ID(),\n\t\t\tName:        req.testCase.Name(),\n\t\t\tType:        req.testCase.Type(),\n\t\t\tGroup:       req.testCase.Group(),\n\t\t\tCapability:  req.testCase.Capability(),\n\t\t\tSuccess:     false,\n\t\t\tUnsupported: isUnsupportedCapabilityError(err),\n\t\t\tError:       err,\n\t\t\tLatency:     latency,\n\t\t}, nil\n\t}\n","sourceCodeStart":283,"sourceCodeEnd":319,"githubUrl":"https://github.com/vxcontrol/pentagi/blob/ea665308baaff015b226f308438a68d929d0f29b/backend/pkg/providers/tester/runner.go#L283-L319","documentation":"executeTest dispatches a test case in priority order: messages-based cases, streaming callback cases, then simple prompt cases. If a test case defines none of these — empty prompt and no messages — the runner cannot construct any LLM call and returns this error instead of executing the request.","triggerScenarios":"A TestCase registered in the registry (builtin or custom) whose Messages() is empty AND Prompt() returns \"\" — executeTest falls through its switch (runner.go:295-303) and hits the default branch. Usually a custom test case implemented with neither field set, or a registry entry with a missing/empty prompt key.","commonSituations":"A hand-written TestCase added to a custom registry where the author set only the name/ID metadata and forgot the prompt; a YAML/JSON registry entry with an empty prompt field; a refactor of a test case that moved the prompt into a field not read by Prompt(); deserialization silently dropping the prompt key due to a tag mismatch.","solutions":["Implement Prompt() (or Messages()) on the custom TestCase to return the actual test prompt.","Fix the registry entry so the prompt/messages field is populated and deserialized with the correct key.","Add a registry-load-time validation that rejects test cases with neither prompt nor messages so the error surfaces at load, not at execution.","Check for struct-tag or YAML/JSON key mismatches that make Prompt() return \"\" after unmarshaling."],"exampleFix":"// before\ntype myCase struct{ id string }\nfunc (c *myCase) Prompt() string { return \"\" } // forgot to set prompt\n// after\ntype myCase struct{ id, prompt string }\nfunc (c *myCase) Prompt() string { return c.prompt } // prompt: \"list the tools available to you\"","handlingStrategy":"validation","validationCode":"// validate test cases at registry load time\nfor _, tc := range cases {\n    if tc.Prompt() == \"\" && len(tc.Messages()) == 0 {\n        return fmt.Errorf(\"test case %q has no prompt or messages\", tc.ID())\n    }\n}","typeGuard":"func testCaseRunnable(tc testdata.TestCase) bool {\n    return tc.Prompt() != \"\" || len(tc.Messages()) > 0\n}","tryCatchPattern":"res, err := executeTest(ctx, req)\nif err != nil && strings.Contains(err.Error(), \"has no prompt or messages\") {\n    return testdata.TestResult{ID: req.testCase.ID(), Name: req.testCase.Name(),\n        Error: fmt.Errorf(\"malformed test case %q: %w\", req.testCase.ID(), err)}\n}","preventionTips":["Always set a prompt (or messages) when authoring a custom TestCase","Validate registry entries eagerly at load time so malformed cases fail before execution","Use struct literals with named fields so an unset prompt is obvious in review","Watch for silent deserialization drops when loading cases from YAML/JSON","Add a unit test per custom TestCase asserting Prompt() or Messages() is non-empty"],"tags":["go","testing","validation"],"backgroundTag":"test-case-missing-prompt","analyzedSha":"ea665308baaff015b226f308438a68d929d0f29b","analyzedAt":"2026-09-01T14:16:31.421Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}