MemPalace/mempalace · error · ValueError

documents length {len(documents)} does not match ids length

Error message

documents length {len(documents)} does not match ids length {n}

What it means

Before any upsert/add into the qdrant backend, _validate_write_batch checks that len(documents) equals len(ids). A mismatch raises ValueError immediately, so a malformed batch can never produce half-written points.

Source

Thrown at mempalace/backends/qdrant.py:229

            continue
        if key == "$or":
            if not any(_matches_where_document(document, clause) for clause in value or []):
                return False
            continue
        raise UnsupportedFilterError(f"where_document operator {key!r} not supported")
    return True


def _validate_write_batch(
    *,
    documents: list[str],
    ids: list[str],
    metadatas: Optional[list[dict]],
    embeddings: Optional[list[list[float]]],
) -> None:
    n = len(ids)
    if len(documents) != n:
        raise ValueError(f"documents length {len(documents)} does not match ids length {n}")
    if metadatas is not None and len(metadatas) != n:
        raise ValueError(f"metadatas length {len(metadatas)} does not match ids length {n}")
    if embeddings is not None and len(embeddings) != n:
        raise ValueError(f"embeddings length {len(embeddings)} does not match ids length {n}")


def _as_vector_array(vector: list[float]) -> np.ndarray:
    arr = np.asarray(vector, dtype=np.float32)
    if arr.ndim != 1 or arr.size == 0:
        raise ValueError("embedding must be a non-empty 1D vector")
    return arr


def _normalize_vectors(embeddings: list[list[float]]) -> tuple[list[list[float]], int]:
    vectors = []
    dims = set()
    for embedding in embeddings:
        arr = _as_vector_array(embedding)

View on GitHub (pinned to 06cb6987f0)

Solutions

Assert len(documents) == len(ids) before the call.
Derive documents, ids, metadatas, embeddings from one list of records via zip(*...).
Add a unit test over your batch builder that asserts equal lengths.

Example fix

# before
col.add(ids=ids, documents=[d for d in docs if d])

# after
records = [(i, d) for i, d in zip(ids, docs) if d]
col.add(ids=[r[0] for r in records], documents=[r[1] for r in records])

Defensive patterns

Strategy: validation

Validate before calling

assert len(documents) == len(ids), f"documents {len(documents)} != ids {len(ids)}"
col.add(ids=ids, documents=documents)

Type guard

def is_valid_write_batch(ids, documents) -> bool:
    return len(documents) == len(ids)

Try / catch

try:
    col.add(ids=ids, documents=documents)
except ValueError as e:
    if "does not match ids length" in str(e):
        logger.error("ingest batch misaligned: %d ids vs %d docs", len(ids), len(documents))
        raise

Prevention

Build batches as a single list of records; derive parallel arrays with zip(*...).
Add a length assert in every ingest helper before calling add/upsert.

When it happens

Trigger: collection.add(documents=["a"], ids=["1","2"]) or upsert where the documents list was built from a different source than the ids list.

Common situations: Zipping/unzipping mismatches; filtering invalid rows out of documents but not ids; off-by-one slices on one array.

Related errors

AI-assisted analysis of MemPalace/mempalace@06cb6987f0 (2026-08-15). Data as JSON: /api/errors/722b8829191d1b8c. Report an issue: GitHub.