{"record":{"id":"ed8faa6e62341c30","repo":"MemPalace/mempalace","slug":"milvus-filters-do-not-support-null-comparisons","errorCode":null,"errorMessage":"Milvus filters do not support null comparisons","messagePattern":"Milvus filters do not support null comparisons","errorType":"exception","errorClass":"UnsupportedFilterError","httpStatus":null,"severity":"error","filePath":"mempalace/backends/milvus.py","lineNumber":93,"sourceCode":"def _utcnow() -> str:\n    return datetime.now(timezone.utc).isoformat()\n\n\ndef milvus_uri_is_server(uri: Optional[str]) -> bool:\n    \"\"\"Return whether ``uri`` targets service-managed Milvus storage.\"\"\"\n    if not uri:\n        return False\n    normalized = uri.strip().lower()\n    return normalized.startswith((\"http://\", \"https://\", \"tcp://\", \"grpc://\"))\n\n\ndef _quote_value(value: Any) -> str:\n    if isinstance(value, bool):\n        return \"true\" if value else \"false\"\n    if isinstance(value, (int, float)):\n        return repr(value)\n    if value is None:\n        raise UnsupportedFilterError(\"Milvus filters do not support null comparisons\")\n    text = str(value).replace(\"\\\\\", \"\\\\\\\\\").replace('\"', '\\\\\"')\n    return f'\"{text}\"'\n\n\ndef _like_value(value: Any) -> str:\n    text = str(value).replace(\"\\\\\", \"\\\\\\\\\").replace('\"', '\\\\\"')\n    return f'\"%{text}%\"'\n\n\ndef _field_name(name: str) -> str:\n    if not isinstance(name, str) or not _FIELD_RE.match(name):\n        raise UnsupportedFilterError(f\"Milvus filter field {name!r} is not a safe identifier\")\n    return name\n\n\ndef _translate_field(field: str, expected: Any) -> str:\n    field = _field_name(field)\n    if isinstance(expected, dict):","sourceCodeStart":75,"sourceCodeEnd":111,"githubUrl":"https://github.com/MemPalace/mempalace/blob/06cb6987f02610784fefbad4b2bd5d026d164ba6/mempalace/backends/milvus.py#L75-L111","documentation":"Raised by _quote_value() in the Milvus backend when a metadata filter value is None. The backend translates the portable Mongo-style where DSL into Milvus filter expression strings, and Milvus filter syntax has no literal for SQL NULL, so any $eq/$ne/$in/etc. operand of None cannot be rendered and is rejected before the query reaches Milvus. It surfaces as UnsupportedFilterError (a BackendError subclass) from any query() call that passes a null filter value.","triggerScenarios":"Calling query/count/delete with where={\"wing\": None}, where={\"room\": {\"$ne\": None}}, or an $in list containing a None element, e.g. where={\"tag\": {\"$in\": [\"a\", None]}}. Any operator path that routes the operand through _quote_value() with value None triggers it.","commonSituations":"Code ported from ChromaDB backend (which tolerates None values), dynamically built filters from optional dict fields that default to None, or JSON metadata where a key was explicitly set to null during ingest.","solutions":["Remove None operands from the where clause before calling query (skip keys whose value is None)","If you need 'field is absent/null' semantics, filter on a sentinel string like \"__unset__\" stored at ingest time","Catch UnsupportedFilterError and re-issue the query without the null predicate if null-checking is optional"],"exampleFix":"// before\nresults = collection.query(query_texts=[q], where={\"wing\": None}, n_results=5)\n\n// after\nwhere = {k: v for k, v in filters.items() if v is not None}\nresults = collection.query(query_texts=[q], where=where or None, n_results=5)","handlingStrategy":"validation","validationCode":"def strip_null_filters(where):\n    if not where:\n        return None\n    out = {}\n    for k, v in where.items():\n        if isinstance(v, dict):\n            cleaned = {op: val for op, val in v.items() if val is not None}\n            if not cleaned:\n                continue\n            if \"$in\" in cleaned or \"$nin\" in cleaned:\n                for op in (\"$in\", \"$nin\"):\n                    if op in cleaned:\n                        cleaned[op] = [x for x in cleaned[op] if x is not None]\n                if any(not cleaned[op] for op in (\"$in\", \"$nin\") if op in cleaned):\n                    continue\n            out[k] = cleaned\n        elif v is not None:\n            out[k] = v\n    return out or None","typeGuard":"def has_no_null_values(where: dict) -> bool:\n    def check(v):\n        if v is None:\n            return False\n        if isinstance(v, dict):\n            return all(check(x) for x in v.values())\n        if isinstance(v, list):\n            return all(check(x) for x in v)\n        return True\n    return all(check(v) for v in where.values())","tryCatchPattern":"from mempalace.backends.base import UnsupportedFilterError\ntry:\n    results = collection.query(query_texts=[q], where=where, n_results=k)\nexcept UnsupportedFilterError as e:\n    if \"null\" in str(e):\n        where = {k: v for k, v in where.items() if v is not None}\n        results = collection.query(query_texts=[q], where=where, n_results=k)\n    else:\n        raise","preventionTips":["Never put None into where clauses; use sentinel strings for 'unset'","Centralize filter construction in one builder that drops nulls","Write a unit test asserting queries with None values are pre-filtered"],"tags":["milvus","filter","query","null-handling"],"backgroundTag":null,"analyzedSha":"06cb6987f02610784fefbad4b2bd5d026d164ba6","analyzedAt":"2026-08-15T03:03:36.213Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}