chroma-core/chroma · error · ValueError

Document length {len(doc_tokens.ids)} is greater than the ma

Error message

Document length {len(doc_tokens.ids)} is greater than the max tokens {self.max_tokens()}

What it means

Before running the ONNX model, __call__ encodes each batch with the MiniLM tokenizer and raises ValueError when len(doc_tokens.ids) > self.max_tokens(), which is hard-coded to 256 for this model. This is a strict limit — unlike some transformers, there is no truncation=True here — because the model's positional embeddings only cover 256 tokens, so longer inputs would either crash the session or produce garbage embeddings.

Source

Thrown at chromadb/utils/embedding_functions/onnx_mini_lm_l6_v2.py:166

        Args:
            documents: The documents to generate embeddings for.
            batch_size: The batch size to use when generating embeddings.

        Returns:
            The embeddings for the documents.
        """
        all_embeddings = []
        for i in range(0, len(documents), batch_size):
            batch = documents[i : i + batch_size]

            # Encode each document separately
            encoded = [self.tokenizer.encode(d) for d in batch]

            # Check if any document exceeds the max tokens
            for doc_tokens in encoded:
                if len(doc_tokens.ids) > self.max_tokens():
                    raise ValueError(
                        f"Document length {len(doc_tokens.ids)} is greater than the max tokens {self.max_tokens()}"
                    )

            input_ids = np.array([e.ids for e in encoded])
            attention_mask = np.array([e.attention_mask for e in encoded])

            onnx_input = {
                "input_ids": np.array(input_ids, dtype=np.int64),
                "attention_mask": np.array(attention_mask, dtype=np.int64),
                "token_type_ids": np.array(
                    [np.zeros(len(e), dtype=np.int64) for e in input_ids],
                    dtype=np.int64,
                ),
            }

            model_output = self.model.run(None, onnx_input)
            last_hidden_state = model_output[0]

View on GitHub (pinned to aecdd12c8a)

Solutions

Chunk documents before add(): split into overlapping ~200-word / ~800-character pieces (e.g. LangChain RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=100)) and add each as its own document
Or truncate defensively: doc[:8000] characters is a rough guard, but tokenizer-based chunking is the correct fix since the limit is in tokens
If you must embed long documents whole, switch to an EF with a larger context window (e.g. OpenAI text-embedding-3, Ollama nomic-embed-text with 8192, or a 512-token sentence-transformers model)

Example fix

// before
collection.add(ids=["doc1"], documents=[open("report.txt").read()])  # >256 tokens -> ValueError

// after
from langchain.text_splitter import RecursiveCharacterTextSplitter
chunks = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=100).split_text(open("report.txt").read())
collection.add(ids=[f"doc1_{i}" for i in range(len(chunks))], documents=chunks)

Defensive patterns

Strategy: validation

Validate before calling

MAX_CHARS = 800  # conservative: ~200 words < 256 word-piece tokens
def fit_chunks(text: str, chunk_size: int = 800, overlap: int = 100):
    if len(text) <= chunk_size:
        return [text]
    return [text[i:i + chunk_size] for i in range(0, len(text), chunk_size - overlap)]
docs = [c for d in raw_docs for c in fit_chunks(d)]

Try / catch

try:
    collection.add(ids=ids, documents=docs)
except ValueError as e:
    if "max tokens" in str(e):
        docs = [c for d in docs for c in fit_chunks(d)]  # split and retry once
        collection.add(ids=new_ids, documents=docs)
    else:
        raise

Prevention

Always run a text splitter (chunk_size ~800 chars with overlap) before add() with this EF
Remember the 256 limit is in word-piece tokens, not words — non-English text inflates faster
Store chunk_index in metadata so you can reconstruct long documents from pieces
For long-document workloads choose an EF with a larger context window

When it happens

Trigger: collection.add(documents=[long_text]) where the HuggingFace tokenizer yields more than 256 tokens (note: word-piece tokens, not words, so ~190+ English words can exceed it); ingesting full web pages, articles, PDFs, transcripts, or code files without chunking; a batch where just one document exceeds the limit fails the whole call.

Common situations: RAG pipelines ingesting raw documents without a text splitter; importing data from another EF that tolerated longer inputs (e.g. 512-token or 8192-token models); non-English text where tokenization inflates token counts (CJK, agglutinative languages); code/JSON blobs that tokenize densely.

Related errors

AI-assisted analysis of chroma-core/chroma@aecdd12c8a (2026-08-16). Data as JSON: /api/errors/5f593cf32b0e535e. Report an issue: GitHub.