MemPalace/mempalace · error · ValueError

{field_name} contains null bytes

Error message

{field_name} contains null bytes

What it means

The event body (or note/metadata-bearing field routed through _sanitize_body) contains a NUL character '\x00'. SQLite text and the verbatim-storage contract cannot safely carry embedded NULs, so the value is rejected rather than silently truncated. This fires before the size check.

Source

Thrown at mempalace/logstream.py:148

def _sanitize_status(value) -> Optional[str]:
    if value is None or value == "":
        return None
    if not isinstance(value, str) or value not in EVENT_STATUSES:
        allowed = ", ".join(sorted(EVENT_STATUSES))
        raise ValueError(f"status={value!r} is not one of: {allowed}")
    return value


def _sanitize_body(value, max_bytes: int, field_name: str = "body") -> str:
    """Validate verbatim payload text. Empty is allowed; ``None`` becomes ''."""
    if value is None:
        return ""
    if not isinstance(value, str):
        raise ValueError(f"{field_name} must be a string")
    if "\x00" in value:
        raise ValueError(f"{field_name} contains null bytes")
    value = strip_lone_surrogates(value)
    size = len(value.encode("utf-8"))
    if size > max_bytes:
        raise ValueError(f"{field_name} is {size} bytes; maximum is {max_bytes} bytes")
    return value


def _sanitize_metadata(value) -> str:
    """Validate optional metadata dict and return its canonical JSON text."""
    if value is None:
        return "{}"
    if not isinstance(value, dict):
        raise ValueError("metadata must be an object")
    try:
        encoded = json.dumps(value, ensure_ascii=False, sort_keys=True)
    except (TypeError, ValueError) as exc:
        raise ValueError(f"metadata is not JSON-serializable: {exc}") from None
    if len(encoded.encode("utf-8")) > MAX_METADATA_BYTES:

View on GitHub (pinned to 06cb6987f0)

Solutions

1. Strip NULs before calling: body = body.replace('\x00', '').
2. Fix the upstream reader to stop at the NUL terminator (e.g. slice to buf.index(b'\x00')).
3. If the payload really is binary, base64-encode it into a text body instead.

Example fix

// before
body = raw.decode("utf-8")  # raw contains b"a\x00b"
evt = ls.append_event(type="log.entry", body=body, ...)
// after
body = raw.decode("utf-8").replace("\x00", "")
evt = ls.append_event(type="log.entry", body=body, ...)

def safe_body(text: str) -> str:
    if "\x00" in text:
        raise ValueError("payload contains NUL bytes; strip or encode before storing")
    return text

# or strip unconditionally when NULs are known padding artifacts:
body = body.replace("\x00", "")

def is_nul_free(text) -> bool:
    return isinstance(text, str) and "\x00" not in text

try:
    evt = ls.append_event(..., body=body)
except ValueError as e:
    if "null bytes" in str(e):
        evt = ls.append_event(..., body=body.replace("\x00", ""))
    else:
        raise

When it happens

Trigger: append_event(body='line1\x00line2'); body built from binary data that was decoded with errors='replace' but still contains NULs; content copied from a fixed-width/padded buffer with NUL padding past the terminator.

Common situations: Reading C-strings or binary protocol output into Python; log lines concatenated from substr() slices in SQLite or from mmap'd files; artifacts produced on Windows with odd encodings.

Related errors

• content contains null bytes
• {field_name} must be a non-empty string
• {field_name} exceeds maximum length of {_MAX_ROUTING_LENGTH}
• {field_name} contains control characters
• type must be a non-empty string

AI-assisted analysis of MemPalace/mempalace@06cb6987f0 (2026-08-15). Data as JSON: /api/errors/fa3e9aeacb7678ae. Report an issue: GitHub.