{"record":{"id":"d2e0bf8173d7e7e5","repo":"MemPalace/mempalace","slug":"metadata-key-key-r-clashes-with-a-reserved-milvu","errorCode":null,"errorMessage":"metadata key {key!r} clashes with a reserved Milvus field","messagePattern":"metadata key (.+?) clashes with a reserved Milvus field","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"mempalace/backends/milvus.py","lineNumber":241,"sourceCode":"    if len(dims) > 1:\n        raise DimensionMismatchError(f\"milvus batch cannot mix embedding dimensions {sorted(dims)}\")\n    return vectors, dims.pop() if dims else 0\n\n\ndef _clean_text(value: Any) -> str:\n    text = \"\" if value is None else str(value)\n    return strip_lone_surrogates(text).replace(\"\\x00\", \"\")\n\n\ndef _utf8_len(value: str) -> int:\n    return len(value.encode(\"utf-8\"))\n\n\ndef _jsonable_metadata(meta: dict | None) -> dict:\n    cleaned = {}\n    for key, value in (meta or {}).items():\n        if key in RESERVED_FIELDS:\n            raise ValueError(f\"metadata key {key!r} clashes with a reserved Milvus field\")\n        try:\n            json.dumps(value, ensure_ascii=False)\n        except (TypeError, ValueError):\n            value = str(value)\n        cleaned[str(key)] = value\n    return cleaned\n\n\ndef _slug(value: str, fallback: str = \"collection\") -> str:\n    safe = re.sub(r\"[^A-Za-z0-9_]+\", \"_\", value).strip(\"_\")\n    if not safe or not re.match(r\"^[A-Za-z_]\", safe):\n        safe = f\"{fallback}_{safe}\" if safe else fallback\n    if len(safe) <= 120:\n        return safe\n    digest = sha256(value.encode(\"utf-8\", errors=\"surrogatepass\")).hexdigest()[:12]\n    return f\"{safe[:107]}_{digest}\"\n\n","sourceCodeStart":223,"sourceCodeEnd":259,"githubUrl":"https://github.com/MemPalace/mempalace/blob/06cb6987f02610784fefbad4b2bd5d026d164ba6/mempalace/backends/milvus.py#L223-L259","documentation":"Raised by _jsonable_metadata() when a metadata dict uses one of the reserved field names: id, document, metadata, vector, sparse, distance, score. These are the Milvus collection's structural columns, so metadata keys cannot shadow them; the backend raises ValueError before insert rather than letting the payload collide. Values that are not JSON-serializable are stringified instead (no error) — only key collisions raise.","triggerScenarios":"add(..., metadatas=[{\"id\": \"x\", \"document\": \"text\", \"score\": 1.5}]) — any of the seven reserved names used as a metadata key.","commonSituations":"Ingesting raw records that carry their own 'id', 'document', or 'score' fields; porting ChromaDB metadata that happened to use reserved words.","solutions":["Prefix or rename reserved keys at ingest: source_id, source_document, relevance_score","Add a metadata sanitizer that maps RESERVED_FIELDS names before calling add","If the collision is intentional (e.g. drawer id), use the backend's dedicated id parameter instead of metadata"],"exampleFix":"# before\ncollection.add(ids=[i], documents=[d], metadatas=[{\"id\": i, \"score\": 0.9}])\n\n# after\ncollection.add(ids=[i], documents=[d], metadatas=[{\"source_id\": i, \"relevance_score\": 0.9}])","handlingStrategy":"validation","validationCode":"RESERVED = {\"id\", \"document\", \"metadata\", \"vector\", \"sparse\", \"distance\", \"score\"}\n\ndef sanitize_metadata(meta: dict) -> dict:\n    return {(f\"src_{k}\" if k in RESERVED else k): v for k, v in (meta or {}).items()}","typeGuard":null,"tryCatchPattern":"try:\n    collection.add(ids=ids, documents=docs, metadatas=metas)\nexcept ValueError as e:\n    if \"reserved Milvus field\" in str(e):\n        metas = [sanitize_metadata(m) for m in metas]\n        collection.add(ids=ids, documents=docs, metadatas=metas)\n    else:\n        raise","preventionTips":["Sanitize incoming record dicts once at ingest; never store reserved names as metadata keys","Add a unit test listing RESERVED_FIELDS asserting your sanitizer remaps all of them","Keep one mapping table of source-field → safe-metadata-key"],"tags":["milvus","metadata","reserved-fields","validation"],"backgroundTag":null,"analyzedSha":"06cb6987f02610784fefbad4b2bd5d026d164ba6","analyzedAt":"2026-08-15T03:03:36.213Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}