{"record":{"id":"e310474dca0dacec","repo":"danielmiessler/Fabric","slug":"pattern-validation-failed-validation-message","errorCode":null,"errorMessage":"Pattern validation failed: {validation_message}","messagePattern":"Pattern validation failed: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"scripts/python_ui/streamlit.py","lineNumber":434,"sourceCode":"\n                # Execute pattern\n                result = run(\n                    cmd, input=content, capture_output=True, text=True, check=True\n                )\n                structured_content = result.stdout.strip()\n\n                if not structured_content:\n                    raise ValueError(\"No output received from create_pattern\")\n\n                # Save the structured content to system.md\n                system_file = os.path.join(new_pattern_path, \"system.md\")\n                with open(system_file, \"w\") as f:\n                    f.write(structured_content)\n\n                # Validate the created pattern\n                is_valid, validation_message = validate_pattern(pattern_name)\n                if not is_valid:\n                    raise ValueError(f\"Pattern validation failed: {validation_message}\")\n\n                logger.info(\n                    f\"Successfully created pattern '{pattern_name}' with structured content\"\n                )\n\n            except CalledProcessError as e:\n                error_msg = f\"Error running create_pattern: {e.stderr}\"\n                logger.error(error_msg)\n                if os.path.exists(new_pattern_path):\n                    shutil.rmtree(new_pattern_path)\n                return False, error_msg\n\n            except Exception as e:\n                error_msg = f\"Unexpected error during content structuring: {str(e)}\"\n                logger.error(error_msg)\n                if os.path.exists(new_pattern_path):\n                    shutil.rmtree(new_pattern_path)\n                return False, error_msg","sourceCodeStart":416,"sourceCodeEnd":452,"githubUrl":"https://github.com/danielmiessler/Fabric/blob/338b89cfe97ab2d12ce30ce8b5449857a841366d/scripts/python_ui/streamlit.py#L416-L452","documentation":"Raised after create_pattern output was written to <pattern>/system.md and validate_pattern(pattern_name) rejected it. validate_pattern checks the created pattern's structure (presence/shape of system.md and possibly fabric.md); failure means the LLM-generated content does not satisfy the expected pattern file layout, or a required companion file is missing. The surrounding handler cleans up the new pattern directory on subprocess errors, so this leaves orphaned/invalid files unless handled.","triggerScenarios":"The model returned prose or a preamble instead of the raw pattern content expected in system.md; the validator expects both system.md and fabric.md but only system.md was written; the model wrapped output in markdown fences the validator rejects; pattern name with characters that make validate_pattern look in the wrong directory.","commonSituations":"Weaker models ignoring the create_pattern output contract, validator rules tightened after patterns were authored, hallucinated front-matter in generated output, case-sensitivity mismatches between the created directory name and what validate_pattern resolves.","solutions":["Read validation_message — it names the exact failed check; inspect the generated system.md in the new pattern directory to see what the model actually produced","If the output is wrapped in code fences or preamble, strip them before writing system.md","Retry with a stronger model that follows the create_pattern format, or paste the content into a hand-authored pattern instead","Ensure this error path also removes new_pattern_path like the CalledProcessError handler does, so invalid patterns don't linger and break later listing","Check that validate_pattern and the creation code agree on the pattern directory and required files"],"exampleFix":"# before\nis_valid, validation_message = validate_pattern(pattern_name)\nif not is_valid:\n    raise ValueError(f\"Pattern validation failed: {validation_message}\")\n\n# after\nis_valid, validation_message = validate_pattern(pattern_name)\nif not is_valid:\n    if os.path.exists(new_pattern_path):\n        shutil.rmtree(new_pattern_path)  # match the CalledProcessError cleanup\n    raise ValueError(f\"Pattern validation failed: {validation_message}\")","handlingStrategy":"validation","validationCode":"# Validate the LLM output BEFORE writing and registering the pattern\nREQUIRED_FILES = (\"system.md\",)\n\ndef looks_like_pattern(text: str) -> bool:\n    return bool(text) and len(text.strip()) > 20 and not text.strip().startswith(\"```{\")\n\nif not looks_like_pattern(structured_content):\n    shutil.rmtree(new_pattern_path, ignore_errors=True)\n    return False, \"create_pattern output does not look like a pattern system prompt\"","typeGuard":"def is_valid_pattern_dir(path: str) -> bool:\n    \"\"\"Structural check matching validate_pattern's expectations.\"\"\"\n    p = Path(path)\n    return p.is_dir() and all((p / f).is_file() and (p / f).stat().st_size > 0 for f in REQUIRED_FILES)","tryCatchPattern":"try:\n    is_valid, validation_message = validate_pattern(pattern_name)\n    if not is_valid:\n        if os.path.exists(new_pattern_path):\n            shutil.rmtree(new_pattern_path)  # keep cleanup symmetric with CalledProcessError path\n        return False, f\"Pattern validation failed: {validation_message}\"\nexcept OSError as e:\n    shutil.rmtree(new_pattern_path, ignore_errors=True)\n    return False, f\"Could not validate pattern: {e}\"","preventionTips":["Strip markdown fences/preamble from model output before writing system.md","Validate structure before registering the pattern so failures never leave half-created directories","Keep validate_pattern and the creation code in the same module so required-file expectations can't drift","On validation failure, show validation_message verbatim — it names the failing check"],"tags":["fabric","pattern","validation","llm-output"],"backgroundTag":null,"analyzedSha":"338b89cfe97ab2d12ce30ce8b5449857a841366d","analyzedAt":"2026-08-15T11:38:51.759Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}