MemPalace/mempalace · error · ValueError

row {idx}: document byte length {document_bytes} exceeds Mil

Error message

row {idx}: document byte length {document_bytes} exceeds Milvus VARCHAR limit {DOCUMENT_MAX_LENGTH}; chunk before storing

What it means

Error "row {idx}: document byte length {document_bytes} exceeds Milvus VARCHAR limit {DOCUMENT_MAX_LENGTH}; chunk before storing" thrown in MemPalace/mempalace.

Source

Thrown at mempalace/backends/milvus.py:516

                f"embeddings length {len(embeddings)} does not match ids length {len(ids)}"
            )
        vectors, dimension = _normalize_vectors(embeddings)
        metadatas = metadatas or [{} for _ in ids]
        rows = []
        for idx, (doc_id, document, metadata, vector) in enumerate(
            zip(ids, documents, metadatas, vectors)
        ):
            if not isinstance(doc_id, str) or not doc_id:
                raise ValueError(f"row {idx}: id must be a non-empty string")
            doc_id_bytes = _utf8_len(doc_id)
            if doc_id_bytes > DRAWER_ID_MAX_LENGTH:
                raise ValueError(
                    f"row {idx}: id byte length {doc_id_bytes} exceeds {DRAWER_ID_MAX_LENGTH}"
                )
            document = _clean_text(document)
            document_bytes = _utf8_len(document)
            if document_bytes > DOCUMENT_MAX_LENGTH:
                raise ValueError(
                    f"row {idx}: document byte length {document_bytes} exceeds "
                    f"Milvus VARCHAR limit {DOCUMENT_MAX_LENGTH}; chunk before storing"
                )
            row = {
                FIELD_ID: doc_id,
                FIELD_DOCUMENT: document,
                FIELD_METADATA: _jsonable_metadata(metadata),
                FIELD_VECTOR: vector,
            }
            row.update(row[FIELD_METADATA])
            rows.append(row)
        return rows, dimension

    def add(self, *, documents, ids, metadatas=None, embeddings=None):
        if embeddings is None:
            raise ValueError("milvus requires explicit embeddings")
        if len(set(ids)) != len(ids):
            raise ValueError("add ids must be unique")

View on GitHub (pinned to 06cb6987f0)

Solutions

1. Chunk the document into smaller pieces before storing so each stays under the Milvus VARCHAR limit

When it happens

Trigger: Thrown at mempalace/backends/milvus.py:516 when the library encounters an invalid state.

Common situations: A single verbatim drawer exceeded the Milvus VARCHAR field capacity; chunking was skipped or mis-sized.

AI-assisted analysis of MemPalace/mempalace@06cb6987f0 (2026-08-15). Data as JSON: /api/errors/b7ad0f9cb7a4301a. Report an issue: GitHub.