{"record":{"id":"3bf475274a6cc031","repo":"headroomlabs-ai/headroom","slug":"smartcrusher-invalid-protected-patterns-regex-p","errorCode":null,"errorMessage":"SmartCrusher: invalid protected_patterns regex {p!r}: {e}","messagePattern":"SmartCrusher: invalid protected_patterns regex (.+?): (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"headroom/transforms/smart_crusher.py","lineNumber":561,"sourceCode":"    # survive the compressed output (never dropped, never marker-only).\n\n    @staticmethod\n    def _compile_protected_patterns(patterns: list[str] | None) -> list[re.Pattern[str]]:\n        \"\"\"Compile `protected_patterns` once at construction time.\n\n        A pattern that fails to compile is a caller bug, not something\n        to swallow — silently treating an invalid regex as \"no rows\n        protected\" would defeat the entire point of audit-safe mode\n        (rows the caller believes are protected wouldn't be).\n        \"\"\"\n        if not patterns:\n            return []\n        compiled = []\n        for p in patterns:\n            try:\n                compiled.append(re.compile(p))\n            except re.error as e:\n                raise ValueError(\n                    f\"SmartCrusher: invalid protected_patterns regex {p!r}: {e}\"\n                ) from e\n        return compiled\n\n    @staticmethod\n    def _canon(item: Any) -> str:\n        \"\"\"Canonical JSON text for a row.\n\n        Used both for protected-pattern matching and for identity\n        comparison across the crush boundary — kept rows are\n        re-serialized by Rust, so rows are matched by content, not by\n        Python object identity.\n        \"\"\"\n        return json.dumps(item, sort_keys=True, default=str)\n\n    def _row_matches_protected(self, item: Any) -> bool:\n        text = self._canon(item)\n        return any(p.search(text) for p in self._protected_patterns)","sourceCodeStart":543,"sourceCodeEnd":579,"githubUrl":"https://github.com/headroomlabs-ai/headroom/blob/322425c43bffde1ed0b64fecf3cf5951565dd82b/headroom/transforms/smart_crusher.py#L543-L579","documentation":"SmartCrusher's audit-safe mode compiles each entry of `protected_patterns` with `re.compile`; a pattern that raises `re.error` is re-raised as ValueError naming the offending pattern. Invalid regexes are treated as caller bugs, not swallowed, because silently treating a broken pattern as \"no protection\" would leave rows the caller believes are protected actually unprotected.","triggerScenarios":"Passing `protected_patterns=[\"[unclosed\"]`, `\"*leading\"`, or any string that fails Python `re.compile` (unbalanced brackets, invalid quantifier placement, bad group syntax) to SmartCrusher compression with protection enabled.","commonSituations":"User-supplied or config-file regexes never validated upstream; patterns written for a different regex flavor (e.g. JS/PCRE lookarounds or possessive quantifiers unsupported by `re`); escaping bugs when building patterns dynamically from strings containing regex metacharacters.","solutions":["Fix the named pattern — the message includes the exact pattern and the underlying `re.error` reason","Test every pattern standalone before passing it: `re.compile(p)` in a startup check or config loader","When a pattern must match literal text (filenames, IDs), use `re.escape(value)` instead of interpolating raw user input into a regex"],"exampleFix":"# before\npatterns = [f\"^client:{client_id}$\"]  # client_id='acme(1)' -> re.error? no, but metachars bite\npatterns = [\"rows[client\"]            # unbalanced bracket -> re.error\n\n# after\nimport re\npatterns = [re.escape(f\"client:{client_id}\")]\nre.compile(patterns[0])  # validate at config-load time, not mid-compression","handlingStrategy":"validation","validationCode":"import re\n\ndef compile_protected_patterns(patterns: list[str]) -> list[re.Pattern]:\n    compiled = []\n    for p in patterns or []:\n        try:\n            compiled.append(re.compile(p))\n        except re.error as e:\n            raise ValueError(f\"invalid protected pattern {p!r}: {e}\") from e\n    return compiled\n\ncompiled = compile_protected_patterns(patterns)  # before SmartCrusher call\ncrusher = SmartCrusher(protected_patterns=patterns)","typeGuard":"def patterns_compile(patterns: list[str]) -> bool:\n    return all(_compiles(p) for p in patterns)\n\ndef _compiles(p: str) -> bool:\n    try:\n        re.compile(p); return True\n    except re.error:\n        return False","tryCatchPattern":"try:\n    result = crusher.compress(items, protected_patterns=patterns)\nexcept ValueError as e:\n    if \"invalid protected_patterns regex\" in str(e):\n        bad = patterns  # message names the pattern; fix source of patterns and abort\n        raise\n    raise","preventionTips":["Compile all patterns at config-load time; never accept unvalidated regexes from user input","Use re.escape() for patterns meant to match literal strings","Unit-test every shipped pattern against re.compile in CI"],"tags":["regex","validation","smart-crusher","protected-patterns"],"backgroundTag":null,"analyzedSha":"322425c43bffde1ed0b64fecf3cf5951565dd82b","analyzedAt":"2026-08-15T01:03:05.481Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}