{"record":{"id":"0450cc87baa88941","repo":"can1357/oh-my-pi","slug":"generated-whentouse-must-start-with-use-this-agen","errorCode":null,"errorMessage":"Generated whenToUse must start with 'Use this agent when...'","messagePattern":"Generated whenToUse must start with 'Use this agent when\\.\\.\\.'","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"packages/coding-agent/src/modes/components/agents-hub.ts","lineNumber":191,"sourceCode":"\tconst parsed = JSON.parse(extractJsonObject(raw)) as Partial<GeneratedAgentSpec>;\n\tif (!parsed || typeof parsed !== \"object\") {\n\t\tthrow new Error(\"Model output is not a JSON object\");\n\t}\n\tif (\n\t\ttypeof parsed.identifier !== \"string\" ||\n\t\ttypeof parsed.whenToUse !== \"string\" ||\n\t\ttypeof parsed.systemPrompt !== \"string\"\n\t) {\n\t\tthrow new Error(\"Model output is missing required fields (identifier, whenToUse, systemPrompt)\");\n\t}\n\tconst identifier = parsed.identifier.trim();\n\tconst whenToUse = parsed.whenToUse.trim();\n\tconst systemPrompt = parsed.systemPrompt.trim();\n\tif (!IDENTIFIER_PATTERN.test(identifier)) {\n\t\tthrow new Error(\"Generated identifier is invalid (must be lowercase kebab-case, 2+ words)\");\n\t}\n\tif (!whenToUse.toLowerCase().startsWith(\"use this agent when\")) {\n\t\tthrow new Error(\"Generated whenToUse must start with 'Use this agent when...'\");\n\t}\n\tif (!systemPrompt) {\n\t\tthrow new Error(\"Generated systemPrompt is empty\");\n\t}\n\treturn { identifier, whenToUse, systemPrompt };\n}\n\nfunction matchAgent(agent: HubAgent, query: string): boolean {\n\tconst text = `${agent.name} ${agent.description} ${SOURCE_LABEL[agent.source]} ${agent.overrideModel ?? \"\"}`;\n\treturn query\n\t\t.trim()\n\t\t.split(/\\s+/)\n\t\t.every(token => fuzzyMatch(token, text).matches);\n}\n\n/**\n * The fullscreen agents hub component. Hosted via\n * `ui.showOverlay(..., { fullscreen: true })`; the host must call","sourceCodeStart":173,"sourceCodeEnd":209,"githubUrl":"https://github.com/can1357/oh-my-pi/blob/969062200754ea02cfac922e5ebb8c608c079e15/packages/coding-agent/src/modes/components/agents-hub.ts#L173-L209","documentation":"parseGeneratedAgentSpec validates the output of an LLM that drafts new agent definitions in the agents hub. The LLM-proposed whenToUse field must begin with the exact phrase 'Use this agent when' (case-insensitive) so generated agent descriptions follow a consistent, recognizable template. If the model returns a description in any other phrasing, this error is thrown and the spec is rejected.","triggerScenarios":"Calling #runAgentCreationArchitect (via the agents hub 'create agent' flow) when the architect LLM returns a whenToUse line that does not start with 'use this agent when' after trimming — e.g. it returns 'Helpful for refactoring...' or 'Best used when...'.","commonSituations":"Weak or non-instructed models ignoring the prompt template; model paraphrasing the description; prompt template updated but validation not; localization or verbose model output wrapping the phrase.","solutions":["Re-run the agent creation flow so the LLM regenerates the spec with the required prefix","Edit the generated description manually so it starts with 'Use this agent when'","Use a stronger model (configure the default/active model pattern) that follows the output template","Relax the validation to accept any non-empty whenToUse if the strict template is not required"],"exampleFix":"// before\nwhenToUse = \"Good for refactoring tasks\"\n// after\nwhenToUse = \"Use this agent when refactoring tasks need automated handling\"","handlingStrategy":"validation","validationCode":"const whenToUse = (parsed.whenToUse ?? \"\").trim();\nif (!whenToUse.toLowerCase().startsWith(\"use this agent when\")) {\n  throw new Error(\"whenToUse must start with 'Use this agent when'\");\n}","typeGuard":"function hasValidWhenToUse(s: unknown): s is { whenToUse: string } {\n  return typeof s === \"object\" && s !== null && typeof (s as any).whenToUse === \"string\" &&\n    (s as any).whenToUse.trim().toLowerCase().startsWith(\"use this agent when\");\n}","tryCatchPattern":"try {\n  const spec = await hub.runAgentCreationArchitect(desc);\n} catch (err) {\n  if (err instanceof Error && err.message.includes(\"whenToUse\")) {\n    spec = { ...manualSpec, whenToUse: `Use this agent when ${desc}` };\n  } else throw err;\n}","preventionTips":["Use a model known to follow strict output templates for the architect","Include explicit format examples in the creation request","Validate LLM output fields before persisting","Keep whenToUse phrasing aligned with the documented template"],"tags":["validation","llm-output","agents"],"backgroundTag":"llm-output-validation-failed","analyzedSha":"969062200754ea02cfac922e5ebb8c608c079e15","analyzedAt":"2026-08-31T10:29:35.737Z","schemaVersion":2},"datasetVersion":"2026-08-31T14:17:45.589Z"}