{"record":{"id":"0c9b60963ac9bbbf","repo":"HKUDS/DeepTutor","slug":"prompt-leak-detected-in-generated-code","errorCode":null,"errorMessage":"prompt leak detected in generated code.","messagePattern":"prompt leak detected in generated code\\.","errorType":"exception","errorClass":"GenerationFailure","httpStatus":null,"severity":"error","filePath":"deeptutor/book/blocks/code.py","lineNumber":96,"sourceCode":"            chapter_title=chapter_title,\n            chapter_summary=chapter_summary or none_label,\n            objectives_inline=\"; \".join(objectives) or none_label,\n            intent=intent,\n            language=language,\n        )\n        data = await llm_json(\n            user_prompt=user_prompt,\n            system_prompt=get_book_prompt(prompts, \"system\"),\n            max_tokens=900,\n            temperature=0.3,\n            language=ctx.language,\n        )\n\n        code = str(data.get(\"code\") or \"\").strip()\n        if not code:\n            raise GenerationFailure(\"LLM did not return any code.\")\n        if \"<think\" in code.lower() or \"</think\" in code.lower():\n            raise GenerationFailure(\"prompt leak detected in generated code.\")\n\n        code_language = str(data.get(\"language\") or language).strip() or language\n        syntax_error = _syntax_error(code, code_language)\n        if syntax_error:\n            # A truncated or malformed snippet is worse than none: the reader\n            # copies it, it fails, and nothing said it was never checked. Fail\n            # the block so the compiler's retry path gets a second attempt.\n            raise GenerationFailure(f\"generated code does not parse: {syntax_error}\")\n\n        metadata = data.get(\"_metadata\") if isinstance(data.get(\"_metadata\"), dict) else {}\n        return (\n            {\n                \"language\": code_language,\n                \"code\": code,\n                \"explanation\": str(data.get(\"explanation\") or \"\").strip(),\n                \"intent\": intent,\n            },\n            [],","sourceCodeStart":78,"sourceCodeEnd":114,"githubUrl":"https://github.com/HKUDS/DeepTutor/blob/3e82f130422a813cdd73c10b21a44e9325f5821a/deeptutor/book/blocks/code.py#L78-L114","documentation":"The generated code contains `<think` / `</think` markup, i.e. chain-of-thought or reasoning leakage from reasoning-style models leaked into the `code` field. The block deliberately rejects it to keep published snippets clean.","triggerScenarios":"Using a reasoning model (or a provider that emits think tags) for the code block; the model wraps its answer in `<think>…</think>` and the extraction didn't strip it, so the literal tags land in `code`.","commonSituations":"Switching the book pipeline's model to a reasoning-tuned endpoint; prompts that encourage step-by-step output; provider changes in how thinking is surfaced.","solutions":["Retry — many providers emit think tags only sporadically","Configure the provider/model to strip or disable thinking output for this call","Use a non-reasoning model for code block generation","Post-process is intentionally NOT done here — the block treats leakage as a failure so the retry path gets a clean attempt"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":"def leaks_reasoning(code: str) -> bool:\n    c = code.lower()\n    return \"<think\" in c or \"</think\" in c","tryCatchPattern":"try:\n    result = await code_gen._generate(ctx)\nexcept GenerationFailure as e:\n    if \"prompt leak\" in str(e):\n        result = await code_gen._generate(ctx)\n    else:\n        raise","preventionTips":["Disable thinking/reasoning output for the code-block model","Strip think tags at the provider layer before returning JSON","Treat leakage as retryable; don't hand-edit generated blocks"],"tags":["llm-output","prompt-leak","reasoning-model","sanitization"],"backgroundTag":"llm-prompt-leak","analyzedSha":"3e82f130422a813cdd73c10b21a44e9325f5821a","analyzedAt":"2026-08-27T06:57:25.364Z","schemaVersion":2},"datasetVersion":"2026-08-27T08:17:20.692Z"}