{"record":{"id":"3cd574d42cdddead","repo":"MemPalace/mempalace","slug":"embedding-must-be-a-non-empty-1d-vector-3cd574","errorCode":null,"errorMessage":"embedding must be a non-empty 1D vector","messagePattern":"embedding must be a non-empty 1D vector","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"mempalace/backends/sqlite_exact.py","lineNumber":75,"sourceCode":"\ndef _json_loads(text: str | None) -> dict:\n    if not text:\n        return {}\n    try:\n        value = json.loads(text)\n    except json.JSONDecodeError:\n        return {}\n    return value if isinstance(value, dict) else {}\n\n\ndef _encode_vector(vector: list[float]) -> bytes:\n    return _as_vector_array(vector).tobytes()\n\n\ndef _as_vector_array(vector: list[float]) -> np.ndarray:\n    arr = np.asarray(vector, dtype=np.float32)\n    if arr.ndim != 1 or arr.size == 0:\n        raise ValueError(\"embedding must be a non-empty 1D vector\")\n    return arr\n\n\ndef _decode_vector(blob: bytes | None) -> list[float]:\n    if not blob:\n        return []\n    return np.frombuffer(blob, dtype=np.float32).astype(float).tolist()\n\n\ndef _decode_array(blob: bytes | None) -> Optional[np.ndarray]:\n    if not blob:\n        return None\n    arr = np.frombuffer(blob, dtype=np.float32)\n    if arr.size == 0:\n        return None\n    return arr\n\n","sourceCodeStart":57,"sourceCodeEnd":93,"githubUrl":"https://github.com/MemPalace/mempalace/blob/06cb6987f02610784fefbad4b2bd5d026d164ba6/mempalace/backends/sqlite_exact.py#L57-L93","documentation":"Raised by the sqlite_exact backend's vector encoder when an embedding passed to add/upsert is not a non-empty 1D sequence of numbers — after conversion to a float32 numpy array it must have ndim == 1 and size > 0. Empty lists, nested lists (2D batches), or None-as-list all fail here before any SQL runs.","triggerScenarios":"add([{..., \"embedding\": []}]) (empty vector), embedding=[[0.1, 0.2]] (batch-shaped/nested), or embedding of a scalar. Also triggered when an embedder returns an empty vector for empty input and callers pass it through unvalidated.","commonSituations":"Embedder dimension/config mismatch producing empty outputs; code that batches documents and accidentally nests the embedding list; skipping an embedding step for some docs and storing an empty placeholder; None coerced to [None] then rejected by float32 conversion paths.","solutions":["Ensure every document's embedding is a flat, non-empty list/tuple of floats matching the collection dimension","Skip or explicitly handle documents without embeddings rather than inserting empty lists","Add a validation helper before insert: all(isinstance(e, (list, tuple)) and len(e) > 0 and not isinstance(e[0], list) for e in embeddings)","Log the offending document id when validation fails to find the upstream embedder bug"],"exampleFix":"# before\ncol.add([{ \"id\": \"d1\", \"text\": \"hello\", \"embedding\": [] }])  # ValueError\n# after\nemb = embedder.encode(\"hello\")\nassert len(emb) == col.dimension\ncol.add([{ \"id\": \"d1\", \"text\": \"hello\", \"embedding\": emb }])","handlingStrategy":"validation","validationCode":"def valid_embedding(e) -> bool:\n    return (\n        isinstance(e, (list, tuple))\n        and len(e) > 0\n        and all(isinstance(x, (int, float)) and not isinstance(x, bool) for x in e)\n    )\n\ndocs = [d for d in docs if valid_embedding(d.get(\"embedding\"))]  # or reject loudly","typeGuard":"def is_flat_nonempty_vector(v) -> bool:\n    \"\"\"True when v serializes to a non-empty 1D float32 vector.\"\"\"\n    return isinstance(v, (list, tuple)) and len(v) > 0 and all(\n        isinstance(x, (int, float)) and not isinstance(x, bool) for x in v\n    )","tryCatchPattern":"try:\n    col.add(docs)\nexcept ValueError as e:\n    if \"non-empty 1D\" in str(e):\n        bad = [d[\"id\"] for d in docs if not is_flat_nonempty_vector(d.get(\"embedding\"))]\n        raise RuntimeError(f\"invalid embeddings for ids: {bad}\") from e\n    raise","preventionTips":["Validate embeddings right after the embedder returns them","Assert embedder output length equals the collection dimension","Never insert placeholder empty vectors; skip or compute embeddings for all docs"],"tags":["sqlite","embedding","validation","valueerror"],"backgroundTag":null,"analyzedSha":"06cb6987f02610784fefbad4b2bd5d026d164ba6","analyzedAt":"2026-08-15T03:03:36.213Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}