{"record":{"id":"07538b649489f9c4","repo":"MemPalace/mempalace","slug":"milvus-filter-field-name-r-is-not-a-safe-identif","errorCode":null,"errorMessage":"Milvus filter field {name!r} is not a safe identifier","messagePattern":"Milvus filter field (.+?) is not a safe identifier","errorType":"exception","errorClass":"UnsupportedFilterError","httpStatus":null,"severity":"error","filePath":"mempalace/backends/milvus.py","lineNumber":105,"sourceCode":"def _quote_value(value: Any) -> str:\n    if isinstance(value, bool):\n        return \"true\" if value else \"false\"\n    if isinstance(value, (int, float)):\n        return repr(value)\n    if value is None:\n        raise UnsupportedFilterError(\"Milvus filters do not support null comparisons\")\n    text = str(value).replace(\"\\\\\", \"\\\\\\\\\").replace('\"', '\\\\\"')\n    return f'\"{text}\"'\n\n\ndef _like_value(value: Any) -> str:\n    text = str(value).replace(\"\\\\\", \"\\\\\\\\\").replace('\"', '\\\\\"')\n    return f'\"%{text}%\"'\n\n\ndef _field_name(name: str) -> str:\n    if not isinstance(name, str) or not _FIELD_RE.match(name):\n        raise UnsupportedFilterError(f\"Milvus filter field {name!r} is not a safe identifier\")\n    return name\n\n\ndef _translate_field(field: str, expected: Any) -> str:\n    field = _field_name(field)\n    if isinstance(expected, dict):\n        parts = []\n        for op, operand in expected.items():\n            if op == \"$eq\":\n                parts.append(f\"{field} == {_quote_value(operand)}\")\n            elif op == \"$ne\":\n                parts.append(f\"{field} != {_quote_value(operand)}\")\n            elif op == \"$in\":\n                if not isinstance(operand, list) or not operand:\n                    raise UnsupportedFilterError(f\"$in requires a non-empty list for {field!r}\")\n                items = \", \".join(_quote_value(item) for item in operand)\n                parts.append(f\"{field} in [{items}]\")\n            elif op == \"$nin\":","sourceCodeStart":87,"sourceCodeEnd":123,"githubUrl":"https://github.com/MemPalace/mempalace/blob/06cb6987f02610784fefbad4b2bd5d026d164ba6/mempalace/backends/milvus.py#L87-L123","documentation":"Raised by _field_name() when a where-clause key fails the identifier regex ^[A-Za-z_][A-Za-z0-9_]*$ or is not a str. Because field names are interpolated directly into the Milvus filter expression string, only safe identifiers are allowed; anything else (dots, dashes, spaces, operators, injection attempts) is rejected with UnsupportedFilterError. This is both a correctness and an injection guard.","triggerScenarios":"where={\"user.name\": \"alice\"} (dotted key), where={\"created-at\": 1}, where={\"1tag\": \"x\"} (leading digit), or a non-string key like where={42: \"v\"} when building the dict from arbitrary data.","commonSituations":"Copying ChromaDB-style metadata keys containing '.' or '-', auto-generating filter keys from user input or file column headers, or keys derived from entity names with hyphens/spaces.","solutions":["Rename metadata keys at ingest time to match [A-Za-z_][A-Za-z0-9_]* (replace '.'/'-' with '_')","If keys come from user input, sanitize them with the same regex before building the where dict","Note values inside $in/$eq are quoted safely — only the field NAME is restricted; store exotic names as values, not keys"],"exampleFix":"// before\nwhere = {\"project.name\": \"mem\"}\ncollection.query(query_texts=[q], where=where)\n\n// after\nwhere = {\"project_name\": \"mem\"}  # key sanitized at ingest and at query\ncollection.query(query_texts=[q], where=where)","handlingStrategy":"validation","validationCode":"import re\nSAFE_FIELD = re.compile(r\"^[A-Za-z_][A-Za-z0-9_]*$\")\n\ndef sanitize_where_keys(where):\n    if not where:\n        return None\n    out = {}\n    for k, v in where.items():\n        safe = re.sub(r\"[^A-Za-z0-9_]\", \"_\", str(k))\n        if not re.match(r\"^[A-Za-z_]\", safe):\n            safe = \"f_\" + safe\n        out[safe] = v\n    return out","typeGuard":"def is_safe_field_name(name) -> bool:\n    return isinstance(name, str) and bool(re.match(r\"^[A-Za-z_][A-Za-z0-9_]*$\", name))","tryCatchPattern":"from mempalace.backends.base import UnsupportedFilterError\ntry:\n    collection.query(query_texts=[q], where=where, n_results=k)\nexcept UnsupportedFilterError as e:\n    if \"safe identifier\" in str(e):\n        raise ValueError(f\"sanitize metadata keys before filtering: {where}\") from e\n    raise","preventionTips":["Apply the same identifier rule when writing metadata at ingest time","Keep a whitelist of queryable metadata keys in your app config","Never build where keys directly from raw user input"],"tags":["milvus","filter","validation","identifier"],"backgroundTag":null,"analyzedSha":"06cb6987f02610784fefbad4b2bd5d026d164ba6","analyzedAt":"2026-08-15T03:03:36.213Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}