MemPalace/mempalace · error · ValueError

metadata exceeds maximum size of {MAX_METADATA_BYTES} bytes

Error message

metadata exceeds maximum size of {MAX_METADATA_BYTES} bytes

What it means

After canonical JSON serialization (ensure_ascii=False, sort_keys=True), the metadata exceeds MAX_METADATA_BYTES = 64 KiB. The limit applies to the encoded text, not the source dict, so Unicode content and key repetition count toward it. Metadata is an annotation layer, so the cap keeps event rows lean; large payloads belong in artifacts.

Source

Thrown at mempalace/logstream.py:167

    value = strip_lone_surrogates(value)
    size = len(value.encode("utf-8"))
    if size > max_bytes:
        raise ValueError(f"{field_name} is {size} bytes; maximum is {max_bytes} bytes")
    return value


def _sanitize_metadata(value) -> str:
    """Validate optional metadata dict and return its canonical JSON text."""
    if value is None:
        return "{}"
    if not isinstance(value, dict):
        raise ValueError("metadata must be an object")
    try:
        encoded = json.dumps(value, ensure_ascii=False, sort_keys=True)
    except (TypeError, ValueError) as exc:
        raise ValueError(f"metadata is not JSON-serializable: {exc}") from None
    if len(encoded.encode("utf-8")) > MAX_METADATA_BYTES:
        raise ValueError(f"metadata exceeds maximum size of {MAX_METADATA_BYTES} bytes")
    return encoded


class Logstream:
    """Durable append-only coordination log (events + artifacts).

    Storage and threading mirror ``KnowledgeGraph``: one SQLite file in
    WAL mode, a per-instance lock around writes, ``check_same_thread=False``
    so the MCP HTTP server can call from worker threads.
    """

    def __init__(
        self,
        db_path: str,
        max_body_bytes: int = DEFAULT_MAX_BODY_BYTES,
        max_artifact_bytes: int = DEFAULT_MAX_ARTIFACT_BYTES,
        replica_id: str = None,
    ):

View on GitHub (pinned to 06cb6987f0)

Solutions

1. Move the large value into put_artifact and keep only a pointer in metadata: metadata={'artifact_id': art['id']}.
2. Trim the metadata to the keys consumers actually read.
3. Check upfront: len(json.dumps(metadata, ensure_ascii=False, sort_keys=True).encode('utf-8')) <= 65536.

Example fix

// before
ls.append_event(..., metadata={"context": huge_text})
// after
art = ls.put_artifact(kind="note", content=huge_text, created_by="mac-codex")
ls.append_event(..., metadata={"context_artifact": art["id"]})

import json
MAX_METADATA_BYTES = 64 * 1024

def metadata_fits(md) -> bool:
    if md is None:
        return True
    encoded = json.dumps(md, ensure_ascii=False, sort_keys=True)
    return len(encoded.encode("utf-8")) <= MAX_METADATA_BYTES

if not metadata_fits(md):
    art = ls.put_artifact(kind="json", content=json.dumps(md), created_by=who)
    md = {"artifact_id": art["id"]}

def is_metadata_within_limit(md) -> bool:
    return md is None or len(json.dumps(md, ensure_ascii=False, sort_keys=True).encode("utf-8")) <= 64 * 1024

try:
    evt = ls.append_event(..., metadata=metadata)
except ValueError as e:
    if "exceeds maximum size" in str(e):
        art = ls.put_artifact(kind="json", content=json.dumps(metadata), created_by=who)
        evt = ls.append_event(..., metadata={"artifact_id": art["id"]})
    else:
        raise

When it happens

Trigger: metadata containing a long embedded string (e.g. {'context': 70_000-char prose}); large lists of ids; ensure_ascii=False means CJK metadata is counted at its real byte width (3 bytes/char) rather than \\uXXXX escapes.

Common situations: Agents stuffing entire prompt/response text into metadata 'for context'; metadata carrying full stack traces or base64 blobs; metadata grown gradually across versions until it crossed 64 KiB.

Related errors

• {field_name} is {size} bytes; maximum is {max_bytes} bytes
• metadata must be an object
• metadata is not JSON-serializable: {exc}
• content is {len(raw)} bytes; maximum is {self.max_artifact_b
• metadata key {key!r} clashes with a reserved Milvus field

AI-assisted analysis of MemPalace/mempalace@06cb6987f0 (2026-08-15). Data as JSON: /api/errors/71ef0da0bf3dd89c. Report an issue: GitHub.