{"record":{"id":"8c2573208c51f880","repo":"MemPalace/mempalace","slug":"content-contains-null-bytes-8c2573","errorCode":null,"errorMessage":"content contains null bytes","messagePattern":"content contains null bytes","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"mempalace/logstream.py","lineNumber":528,"sourceCode":"        created_by: str,\n        metadata: dict = None,\n    ) -> dict:\n        \"\"\"Store exact artifact content (v1: UTF-8 text only).\n\n        Returns the artifact record without echoing ``content`` back —\n        callers already hold the content; readers use :meth:`get_artifact`.\n        For ``kind=patch``, a ``warnings`` list is included when the diff\n        looks unappliable (missing trailing newline, CRLF endings); the\n        content itself is still stored verbatim.\n        \"\"\"\n        if not isinstance(kind, str) or kind not in ARTIFACT_KINDS:\n            allowed = \", \".join(sorted(ARTIFACT_KINDS))\n            raise ValueError(f\"kind={kind!r} is not one of: {allowed}\")\n        created_by = _sanitize_routing(created_by, \"created_by\")\n        if not isinstance(content, str) or not content:\n            raise ValueError(\"content must be a non-empty string\")\n        if \"\\x00\" in content:\n            raise ValueError(\"content contains null bytes\")\n        content = strip_lone_surrogates(content)\n        raw = content.encode(\"utf-8\")\n        if len(raw) > self.max_artifact_bytes:\n            raise ValueError(\n                f\"content is {len(raw)} bytes; maximum is {self.max_artifact_bytes} bytes\"\n            )\n        metadata_json = _sanitize_metadata(metadata)\n\n        artifact_id = _new_id(\"art\")\n        created_at = _utc_now_iso()\n        digest = sha256(raw).hexdigest()\n\n        with self._lock:\n            conn = self._conn()\n            with conn:\n                conn.execute(\n                    \"INSERT INTO artifacts (id, kind, sha256, size_bytes, content,\"\n                    \" created_by, created_at, metadata_json, origin_replica)\"","sourceCodeStart":510,"sourceCodeEnd":546,"githubUrl":"https://github.com/MemPalace/mempalace/blob/06cb6987f02610784fefbad4b2bd5d026d164ba6/mempalace/logstream.py#L510-L546","documentation":"put_artifact rejects content containing NUL characters ('\\\\x00'). Artifacts are stored as exact UTF-8 text and verified by sha256, and embedded NULs corrupt text columns and digest round-trips, so the write fails loudly rather than truncating. This check runs before surrogate stripping and size measurement.","triggerScenarios":"put_artifact(kind='file', content=data.decode('utf-8')) where data has embedded NULs (binary misread as text); fixed-width buffers with NUL padding; protocol captures containing length-prefixed frames with NUL separators.","commonSituations":"Accidentally classifying binary as a text artifact (e.g. .db, .png mislabeled kind='file'); reading C structs via struct.unpack and joining raw bytes; Windows UTF-16 text decoded permissively leaving NULs.","solutions":["Detect binary first and reject/handle: if b'\\\\x00' in raw: treat as binary (v1 has no binary artifact kind).","Strip NULs if they are padding artifacts: content=content.replace('\\\\x00', '').","Base64-encode truly binary payloads into a 'note'/'json' artifact and record the encoding in metadata."],"exampleFix":"// before\nart = ls.put_artifact(kind=\"file\", content=raw.decode(\"utf-8\", errors=\"replace\"), ...)\n// after\nimport base64\nart = ls.put_artifact(kind=\"json\", content=json.dumps({\"encoding\": \"base64\", \"data\": base64.b64encode(raw).decode()}), ...)","handlingStrategy":"validation","validationCode":"def is_text_payload(raw: bytes) -> bool:\n    return b\"\\x00\" not in raw\n\nif not is_text_payload(raw):\n    raise TypeError(\"binary payload; encode as base64 in a json/note artifact\")\ncontent = raw.decode(\"utf-8\")","typeGuard":"def is_nul_free_text(content) -> bool:\n    return isinstance(content, str) and \"\\x00\" not in content","tryCatchPattern":"try:\n    art = ls.put_artifact(kind=kind, content=content, ...)\nexcept ValueError as e:\n    if \"null bytes\" in str(e):\n        art = ls.put_artifact(kind=\"json\", content=json.dumps({\"encoding\": \"base64\", \"data\": __import__(\"base64\").b64encode(content.encode()).decode()}), ...)\n    else:\n        raise","preventionTips":["Check for NUL bytes when reading any file/pipe destined for an artifact; treat NULs as a binary signal.","Keep a single is_binary(data) helper and route binary payloads to base64+json instead of text kinds."],"tags":["validation","logstream","artifact","null-bytes"],"backgroundTag":null,"analyzedSha":"06cb6987f02610784fefbad4b2bd5d026d164ba6","analyzedAt":"2026-08-15T03:03:36.213Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}