chroma-core/chroma · error · ValueError

Sparse embedding function returned unexpected number of embe

Error message

Sparse embedding function returned unexpected number of embeddings.

What it means

During add/upsert, Chroma computes sparse embeddings for records by collecting the inputs from a record-column source field (e.g. documents/uris designated as the sparse source), calling the configured SparseEmbeddingFunction, and writing one embedding per record into the target metadata key. If the function returns a different number of embeddings than inputs, the positional zip would silently misassign vectors, so it raises instead.

Source

Thrown at chromadb/api/models/CollectionCommon.py:669

                    # Get document at this position
                    if idx < len(documents_list):
                        doc = documents_list[idx]
                        if isinstance(doc, str):
                            inputs.append(doc)
                            positions.append(idx)

                # Generate embeddings for all collected documents
                if len(inputs) == 0:
                    continue

                sparse_embeddings = self._sparse_embed(
                    input=inputs,
                    sparse_embedding_function=embedding_func,
                )

                if len(sparse_embeddings) != len(positions):
                    raise ValueError(
                        "Sparse embedding function returned unexpected number of embeddings."
                    )

                for position, embedding in zip(positions, sparse_embeddings):
                    updated_metadatas[position][target_key] = embedding

                continue  # Skip the metadata-based logic below

            # Handle normal case: source_key is a metadata field
            for idx, metadata in enumerate(updated_metadatas):
                if target_key in metadata:
                    continue

                source_value = metadata.get(source_key)
                if not isinstance(source_value, str):
                    continue

                inputs.append(source_value)

View on GitHub (pinned to aecdd12c8a)

Solutions

1. Make the sparse function return exactly one SparseEmbedding per input, same order: `return list_of_embeddings # len == len(input)`
2. Unit-test the function with n inputs and assert `len(out) == n` before wiring it into the schema
3. Do not deduplicate, filter, or batch inside the function; keep it a pure 1:1 map

Example fix

# before
class MySparse:
    def __call__(self, input):
        vec = self._model.encode(input[0])       # only first input
        return [to_sparse(vec)]                  # len 1 for n inputs

# after
class MySparse:
    def __call__(self, input):
        return [to_sparse(self._model.encode(t)) for t in input]  # 1:1 with input

out = sparse_fn(sample_inputs)
assert len(out) == len(sample_inputs), f"sparse fn returned {len(out)} for {len(sample_inputs)} inputs"

def checked_sparse_fn(fn):
    def wrapped(input):
        out = fn(input)
        if len(out) != len(input):
            raise AssertionError(f"sparse EF returned {len(out)} for {len(input)} inputs")
        return out
    return wrapped

When it happens

Trigger: A custom SparseEmbeddingFunction whose __call__/embed_documents returns fewer/more embeddings than the number of collected input records — e.g. returning a single embedding, batching/deduplicating internally, or dropping empty inputs.

Common situations: Wrapping a sparse model (SPLADE, BGE-M3, etc.) that returns a matrix you slice incorrectly, or adapting a dense embedding function signature to the sparse interface.

Related errors

• Sparse embedding function returned unexpected number of embe
• Embedding function returned unexpected number of embeddings
• embeddings and documents cannot both be undefined
• Failed to generate embeddings for your request.
• ids, embeddings, metadatas, and documents must all be the sa

AI-assisted analysis of chroma-core/chroma@aecdd12c8a (2026-08-16). Data as JSON: /api/errors/18c2b80e95bf8752. Report an issue: GitHub.