{"record":{"id":"18c2b80e95bf8752","repo":"chroma-core/chroma","slug":"sparse-embedding-function-returned-unexpected-numb-18c2b8","errorCode":null,"errorMessage":"Sparse embedding function returned unexpected number of embeddings.","messagePattern":"Sparse embedding function returned unexpected number of embeddings\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"chromadb/api/models/CollectionCommon.py","lineNumber":669,"sourceCode":"\n                    # Get document at this position\n                    if idx < len(documents_list):\n                        doc = documents_list[idx]\n                        if isinstance(doc, str):\n                            inputs.append(doc)\n                            positions.append(idx)\n\n                # Generate embeddings for all collected documents\n                if len(inputs) == 0:\n                    continue\n\n                sparse_embeddings = self._sparse_embed(\n                    input=inputs,\n                    sparse_embedding_function=embedding_func,\n                )\n\n                if len(sparse_embeddings) != len(positions):\n                    raise ValueError(\n                        \"Sparse embedding function returned unexpected number of embeddings.\"\n                    )\n\n                for position, embedding in zip(positions, sparse_embeddings):\n                    updated_metadatas[position][target_key] = embedding\n\n                continue  # Skip the metadata-based logic below\n\n            # Handle normal case: source_key is a metadata field\n            for idx, metadata in enumerate(updated_metadatas):\n                if target_key in metadata:\n                    continue\n\n                source_value = metadata.get(source_key)\n                if not isinstance(source_value, str):\n                    continue\n\n                inputs.append(source_value)","sourceCodeStart":651,"sourceCodeEnd":687,"githubUrl":"https://github.com/chroma-core/chroma/blob/aecdd12c8a891610db8653630b066b32ceb678b5/chromadb/api/models/CollectionCommon.py#L651-L687","documentation":"During add/upsert, Chroma computes sparse embeddings for records by collecting the inputs from a record-column source field (e.g. documents/uris designated as the sparse source), calling the configured SparseEmbeddingFunction, and writing one embedding per record into the target metadata key. If the function returns a different number of embeddings than inputs, the positional zip would silently misassign vectors, so it raises instead.","triggerScenarios":"A custom SparseEmbeddingFunction whose __call__/embed_documents returns fewer/more embeddings than the number of collected input records — e.g. returning a single embedding, batching/deduplicating internally, or dropping empty inputs.","commonSituations":"Wrapping a sparse model (SPLADE, BGE-M3, etc.) that returns a matrix you slice incorrectly, or adapting a dense embedding function signature to the sparse interface.","solutions":["Make the sparse function return exactly one SparseEmbedding per input, same order: `return list_of_embeddings  # len == len(input)`","Unit-test the function with n inputs and assert `len(out) == n` before wiring it into the schema","Do not deduplicate, filter, or batch inside the function; keep it a pure 1:1 map"],"exampleFix":"# before\nclass MySparse:\n    def __call__(self, input):\n        vec = self._model.encode(input[0])       # only first input\n        return [to_sparse(vec)]                  # len 1 for n inputs\n\n# after\nclass MySparse:\n    def __call__(self, input):\n        return [to_sparse(self._model.encode(t)) for t in input]  # 1:1 with input","handlingStrategy":"type-guard","validationCode":"out = sparse_fn(sample_inputs)\nassert len(out) == len(sample_inputs), f\"sparse fn returned {len(out)} for {len(sample_inputs)} inputs\"","typeGuard":"def checked_sparse_fn(fn):\n    def wrapped(input):\n        out = fn(input)\n        if len(out) != len(input):\n            raise AssertionError(f\"sparse EF returned {len(out)} for {len(input)} inputs\")\n        return out\n    return wrapped","tryCatchPattern":null,"preventionTips":["Unit-test custom sparse functions for 1, 2, and n inputs before wiring into a schema","Keep embedding functions pure 1:1 maps — no dedupe, filter, or internal batching","Wrap third-party sparse models with a length-asserting adapter"],"tags":["sparse-embeddings","custom-embedding-function","count-mismatch","upsert"],"backgroundTag":"embedding-count-mismatch","analyzedSha":"aecdd12c8a891610db8653630b066b32ceb678b5","analyzedAt":"2026-08-16T21:53:27.228Z","schemaVersion":2},"datasetVersion":"2026-08-16T23:17:17.608Z"}