MemPalace/mempalace · error · UnsupportedFilterError

Milvus filter field {name!r} is not a safe identifier

Error message

Milvus filter field {name!r} is not a safe identifier

What it means

Raised by _field_name() when a where-clause key fails the identifier regex ^[A-Za-z_][A-Za-z0-9_]*$ or is not a str. Because field names are interpolated directly into the Milvus filter expression string, only safe identifiers are allowed; anything else (dots, dashes, spaces, operators, injection attempts) is rejected with UnsupportedFilterError. This is both a correctness and an injection guard.

Source

Thrown at mempalace/backends/milvus.py:105

def _quote_value(value: Any) -> str:
    if isinstance(value, bool):
        return "true" if value else "false"
    if isinstance(value, (int, float)):
        return repr(value)
    if value is None:
        raise UnsupportedFilterError("Milvus filters do not support null comparisons")
    text = str(value).replace("\\", "\\\\").replace('"', '\\"')
    return f'"{text}"'


def _like_value(value: Any) -> str:
    text = str(value).replace("\\", "\\\\").replace('"', '\\"')
    return f'"%{text}%"'


def _field_name(name: str) -> str:
    if not isinstance(name, str) or not _FIELD_RE.match(name):
        raise UnsupportedFilterError(f"Milvus filter field {name!r} is not a safe identifier")
    return name


def _translate_field(field: str, expected: Any) -> str:
    field = _field_name(field)
    if isinstance(expected, dict):
        parts = []
        for op, operand in expected.items():
            if op == "$eq":
                parts.append(f"{field} == {_quote_value(operand)}")
            elif op == "$ne":
                parts.append(f"{field} != {_quote_value(operand)}")
            elif op == "$in":
                if not isinstance(operand, list) or not operand:
                    raise UnsupportedFilterError(f"$in requires a non-empty list for {field!r}")
                items = ", ".join(_quote_value(item) for item in operand)
                parts.append(f"{field} in [{items}]")
            elif op == "$nin":

View on GitHub (pinned to 06cb6987f0)

Solutions

1. Rename metadata keys at ingest time to match [A-Za-z_][A-Za-z0-9_]* (replace '.'/'-' with '_')
2. If keys come from user input, sanitize them with the same regex before building the where dict
3. Note values inside $in/$eq are quoted safely — only the field NAME is restricted; store exotic names as values, not keys

Example fix

// before
where = {"project.name": "mem"}
collection.query(query_texts=[q], where=where)

// after
where = {"project_name": "mem"}  # key sanitized at ingest and at query
collection.query(query_texts=[q], where=where)

import re
SAFE_FIELD = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")

def sanitize_where_keys(where):
    if not where:
        return None
    out = {}
    for k, v in where.items():
        safe = re.sub(r"[^A-Za-z0-9_]", "_", str(k))
        if not re.match(r"^[A-Za-z_]", safe):
            safe = "f_" + safe
        out[safe] = v
    return out

def is_safe_field_name(name) -> bool:
    return isinstance(name, str) and bool(re.match(r"^[A-Za-z_][A-Za-z0-9_]*$", name))

from mempalace.backends.base import UnsupportedFilterError
try:
    collection.query(query_texts=[q], where=where, n_results=k)
except UnsupportedFilterError as e:
    if "safe identifier" in str(e):
        raise ValueError(f"sanitize metadata keys before filtering: {where}") from e
    raise

When it happens

Trigger: where={"user.name": "alice"} (dotted key), where={"created-at": 1}, where={"1tag": "x"} (leading digit), or a non-string key like where={42: "v"} when building the dict from arbitrary data.

Common situations: Copying ChromaDB-style metadata keys containing '.' or '-', auto-generating filter keys from user input or file column headers, or keys derived from entity names with hyphens/spaces.

Related errors

• $in requires a non-empty list for {field!r}
• $nin requires a non-empty list for {field!r}
• Milvus filters do not support null comparisons
• operator {op!r} not supported by milvus
• where clause must be a dict, got {type(clause).__name__}

AI-assisted analysis of MemPalace/mempalace@06cb6987f0 (2026-08-15). Data as JSON: /api/errors/07538b649489f9c4. Report an issue: GitHub.