{"record":{"id":"0ca21ee2399e72f0","repo":"chroma-core/chroma","slug":"sparsevector-indices-must-be-sorted-in-strictly-as","errorCode":null,"errorMessage":"SparseVector indices must be sorted in strictly ascending order, found indices[{i}]={self.indices[i]} <= indices[{i-1}]={self.indices[i-1]}","messagePattern":"SparseVector indices must be sorted in strictly ascending order, found indices\\[(.+?)\\]=(.+?) <= indices\\[(.+?)\\]=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"chromadb/base_types.py","lineNumber":81,"sourceCode":"                raise ValueError(\n                    f\"SparseVector indices must be integers, got {type(idx).__name__} at position {i}\"\n                )\n            if idx < 0:\n                raise ValueError(\n                    f\"SparseVector indices must be non-negative, got {idx} at position {i}\"\n                )\n\n        for i, val in enumerate(self.values):\n            if not isinstance(val, (int, float)):\n                raise ValueError(\n                    f\"SparseVector values must be numbers, got {type(val).__name__} at position {i}\"\n                )\n\n        # Validate indices are sorted in strictly ascending order\n        if len(self.indices) > 1:\n            for i in range(1, len(self.indices)):\n                if self.indices[i] <= self.indices[i - 1]:\n                    raise ValueError(\n                        f\"SparseVector indices must be sorted in strictly ascending order, \"\n                        f\"found indices[{i}]={self.indices[i]} <= indices[{i-1}]={self.indices[i-1]}\"\n                    )\n\n    def to_dict(self) -> Dict[str, Any]:\n        \"\"\"Serialize to transport format with type tag.\n\n        Note: Uses 'tokens' as the wire format key name for compatibility\n        with the protobuf schema, even though the Python attribute is 'labels'.\n        \"\"\"\n        result = {\n            TYPE_KEY: SPARSE_VECTOR_TYPE_VALUE,\n            \"indices\": self.indices,\n            \"values\": self.values,\n        }\n        if self.labels is not None:\n            result[\"tokens\"] = self.labels  # Wire format uses 'tokens'\n        return result","sourceCodeStart":63,"sourceCodeEnd":99,"githubUrl":"https://github.com/chroma-core/chroma/blob/aecdd12c8a891610db8653630b066b32ceb678b5/chromadb/base_types.py#L63-L99","documentation":"SparseVector.__post_init__ enforces that indices are sorted in strictly ascending order - sorted and free of duplicates. Chroma's sparse matching code relies on this canonical form, and a duplicate index would make the index-to-value mapping ambiguous. The message pinpoints the first offending pair (indices[i] <= indices[i-1]), i.e. either an out-of-order entry or a duplicate.","triggerScenarios":"Aggregating per-token entries by appending (e.g. bag-of-words where the same token appears twice, producing duplicate indices); concatenating two sparse vectors without merging; building indices from a dict/Counter without sorting; IDs inserted in document order instead of dimension order.","commonSituations":"Count-based sparse embeddings built by looping over tokens; unions of sparse vectors done via list concatenation; vocab IDs assumed sorted but aren't.","solutions":["Build a dict index->value first (summing duplicates), then emit sorted: merged[i] = merged.get(i, 0.0) + v; indices = sorted(merged); values = [merged[i] for i in indices].","If duplicates are impossible by construction, sort pairs: pairs = sorted(zip(indices, values)) and construct from that.","Decide the duplicate policy explicitly (sum weights, take max, or keep first) instead of relying on input order."],"exampleFix":"// before\nsv = SparseVector(indices=[5, 3, 5], values=[0.2, 0.1, 0.3])  # ValueError: indices[2]=5 <= indices[1]=3\n\n// after\nmerged = {}\nfor i, v in zip([5, 3, 5], [0.2, 0.1, 0.3]):\n    merged[i] = merged.get(i, 0.0) + v\nsv = SparseVector(indices=sorted(merged), values=[merged[i] for i in sorted(merged)])","handlingStrategy":"validation","validationCode":"def canonicalize(indices, values):\n    \"\"\"Merge duplicate indices (summing values) and sort ascending.\"\"\"\n    merged = {}\n    for i, v in zip(indices, values):\n        merged[i] = merged.get(i, 0.0) + v\n    order = sorted(merged)\n    return order, [merged[i] for i in order]\n\nindices, values = canonicalize(indices, values)\nsv = SparseVector(indices=indices, values=values)","typeGuard":"def is_strictly_ascending(xs) -> bool:\n    return all(b > a for a, b in zip(xs, xs[1:]))","tryCatchPattern":"try:\n    sv = SparseVector(indices=indices, values=values)\nexcept ValueError as e:\n    raise ValueError(f'sparse embedding not canonical for doc {doc_id}: {e}') from e","preventionTips":["Accumulate into dict[index -> value] while building the embedding; sort once at the end.","Never concatenate two sparse vectors' lists - merge their dicts instead.","Assert is_strictly_ascending(indices) in debug builds of your embedder."],"tags":["sparse-vector","sort-order","duplicates"],"backgroundTag":"sparse-vector-validation","analyzedSha":"aecdd12c8a891610db8653630b066b32ceb678b5","analyzedAt":"2026-08-16T21:53:27.228Z","schemaVersion":2},"datasetVersion":"2026-08-16T23:17:17.608Z"}