bytedance/deer-flow · error · ValueError

retrieval fact.id must be a non-empty string

Error message

retrieval fact.id must be a non-empty string

What it means

ValueError from FTS5RetrievalAdapter._document: a fact dict passed for indexing has an 'id' that is not a non-empty string (missing, None, empty, or non-str). The id becomes part of the JSON composite document key (scope_user, scope_agent, fact_id), so it must be a stable non-empty string.

Source

Thrown at backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/retrieval.py:574

    The index is derived data. Canonical facts remain in Markdown and storage
    notifications update only the addressed row. A deterministic composite
    document id prevents equal fact ids in two user/agent scopes from
    overwriting each other.
    """

    def __init__(self, db_path: str | Path = ":memory:") -> None:
        self._engine = FTS5Retrieval(db_path)

    @staticmethod
    def _document_id(fact_id: str, scope: dict[str, str | None]) -> str:
        scope_user, scope_agent = _scope_key(scope)
        return json.dumps([scope_user, scope_agent, fact_id], ensure_ascii=False, separators=(",", ":"))

    def _document(self, fact: dict[str, Any], scope: dict[str, str | None]) -> dict[str, Any]:
        fact_id = fact.get("id")
        content = fact.get("content")
        if not isinstance(fact_id, str) or not fact_id:
            raise ValueError("retrieval fact.id must be a non-empty string")
        if not isinstance(content, str) or not content.strip():
            raise ValueError("retrieval fact.content must be a non-empty string")
        scope_user, scope_agent = _scope_key(scope)
        payload = dict(fact)
        payload["scope"] = {"userId": scope.get("userId"), "agentName": scope.get("agentName")}
        source = payload.get("source")
        return {
            "fact_id": self._document_id(fact_id, scope),
            "content": content,
            "category": str(payload.get("category") or "context"),
            "confidence": float(payload.get("confidence") or 0.5),
            "created_at": payload.get("createdAt") if isinstance(payload.get("createdAt"), str) else None,
            "scope_user": scope_user,
            "scope_agent": scope_agent,
            "source": source if isinstance(source, str) else json.dumps(source, ensure_ascii=False, default=str),
            "fact_data": payload,
        }

View on GitHub (pinned to 1dd6ba1acb)

Solutions

1. Ensure every fact carries a unique non-empty string id before indexing (generate one if missing: uuid4 hex).
2. Map upstream field names to 'id' explicitly when converting rows to fact dicts.
3. Validate the fact batch before submission: reject/log entries failing the id check.

Example fix

# before
adapter.upsert({"content": "user lives in Berlin"}, scope)  # no id

# after
import uuid
adapter.upsert({"id": uuid.uuid4().hex, "content": "user lives in Berlin"}, scope)

def is_indexable_fact(fact) -> bool:
    return isinstance(fact, dict) and isinstance(fact.get('id'), str) and bool(fact['id'].strip())

def is_indexable_fact(fact: dict) -> bool:
    fid = fact.get('id') if isinstance(fact, dict) else None
    return isinstance(fid, str) and len(fid) > 0

try:
    adapter.upsert(fact, scope)
except ValueError as e:
    if 'fact.id' in str(e):
        logger.warning('skipping fact without usable id: %r', fact)
        continue  # bad fact must not kill the whole indexing batch
    raise

When it happens

Trigger: Calling the adapter's indexing/upsert path with facts like {"content": "..."} (no id), {"id": "", ...}, or {"id": 42, ...}.

Common situations: Facts extracted by the memory pipeline whose id field was never populated; id named differently (fact_id, key) and not mapped; None id from a failed upstream generation; numeric ids from a database row passed raw.

Related errors

• retrieval fact.content must be a non-empty string
• retrieval scope userId must be a string or null
• retrieval scope agentName must be a string or null
• unsupported FTS5 retrieval mode: {mode}
• guaranteed_categories must be an iterable of strings, not a

AI-assisted analysis of bytedance/deer-flow@1dd6ba1acb (2026-08-14). Data as JSON: /api/errors/10030e2f175cceef. Report an issue: GitHub.