{"record":{"id":"f72690928c4913ed","repo":"chroma-core/chroma","slug":"batch-size-must-be-less-than-or-equal-to-sync-thre","errorCode":null,"errorMessage":"batch_size must be less than or equal to sync_threshold","messagePattern":"batch_size must be less than or equal to sync_threshold","errorType":"validation","errorClass":"InvalidConfigurationError","httpStatus":null,"severity":"error","filePath":"chromadb/api/configuration.py","lineNumber":286,"sourceCode":"        \"sync_threshold\": ConfigurationDefinition(\n            name=\"sync_threshold\",\n            validator=lambda value: isinstance(value, int) and value >= 1,\n            is_static=True,\n            default_value=1000,\n        ),\n    }\n\n    @override\n    def configuration_validator(self) -> None:\n        batch_size = self.parameter_map.get(\"batch_size\")\n        sync_threshold = self.parameter_map.get(\"sync_threshold\")\n\n        if (\n            batch_size\n            and sync_threshold\n            and cast(int, batch_size.value) > cast(int, sync_threshold.value)\n        ):\n            raise InvalidConfigurationError(\n                \"batch_size must be less than or equal to sync_threshold\"\n            )\n\n    @classmethod\n    def from_legacy_params(cls, params: Dict[str, Any]) -> Self:\n        \"\"\"Returns an HNSWConfiguration from a metadata dict containing legacy HNSW parameters. Used for migration.\"\"\"\n\n        # We maintain this map to avoid a circular import with HnswParams, and\n        # because then names won't change since we intend to deprecate HNSWParams\n        # in favor of this type of configuration.\n        old_to_new = {\n            \"hnsw:space\": \"space\",\n            \"hnsw:construction_ef\": \"ef_construction\",\n            \"hnsw:search_ef\": \"ef_search\",\n            \"hnsw:M\": \"M\",\n            \"hnsw:num_threads\": \"num_threads\",\n            \"hnsw:resize_factor\": \"resize_factor\",\n            \"hnsw:batch_size\": \"batch_size\",","sourceCodeStart":268,"sourceCodeEnd":304,"githubUrl":"https://github.com/chroma-core/chroma/blob/aecdd12c8a891610db8653630b066b32ceb678b5/chromadb/api/configuration.py#L268-L304","documentation":"HNSWConfigurationInternal.configuration_validator (chromadb/api/configuration.py:286) raises InvalidConfigurationError when batch_size exceeds sync_threshold. This cross-parameter constraint exists because HNSW index maintenance batches inserts and syncs the index at sync_threshold; a batch larger than the sync threshold would break the batching/syncing contract. It runs at the end of construction, after individual parameter validation passes.","triggerScenarios":"Creating or updating an HNSW configuration with batch_size > sync_threshold, e.g. HNSWConfigurationInterface(batch_size=2000, sync_threshold=1000); migrating legacy collection metadata (hnsw:batch_size / hnsw:sync_threshold) that carries the same violation; loading stored config JSON with an inconsistent pair.","commonSituations":"Tuning for throughput: raising batch_size without raising sync_threshold; legacy collections created by older versions with looser or no enforcement, now being migrated; mixing values from different tuning guides for the two knobs.","solutions":["Raise sync_threshold to at least batch_size (or lower batch_size) so the invariant batch_size <= sync_threshold holds","When migrating legacy params, validate and adjust the pair before constructing via from_legacy_params","For very large batches, scale both together, e.g. batch_size=10000 with sync_threshold=10000"],"exampleFix":"# before\ncfg = HNSWConfigurationInterface(batch_size=2000, sync_threshold=1000)  # InvalidConfigurationError\n# after\ncfg = HNSWConfigurationInterface(batch_size=1000, sync_threshold=2000)  # invariant holds","handlingStrategy":"validation","validationCode":"def validate_hnsw_pair(batch_size: int, sync_threshold: int) -> None:\n    if batch_size > sync_threshold:\n        raise ValueError(\n            f\"batch_size ({batch_size}) must be <= sync_threshold ({sync_threshold})\"\n        )\n\nvalidate_hnsw_pair(batch_size, sync_threshold)\ncfg = HNSWConfigurationInterface(batch_size=batch_size, sync_threshold=sync_threshold)","typeGuard":null,"tryCatchPattern":"from chromadb.api.configuration import InvalidConfigurationError\n\ntry:\n    cfg = HNSWConfigurationInterface(batch_size=b, sync_threshold=s)\nexcept InvalidConfigurationError:\n    cfg = HNSWConfigurationInterface(batch_size=min(b, s), sync_threshold=s)  # clamp batch_size","preventionTips":["Always set batch_size and sync_threshold together, keeping batch_size <= sync_threshold","Scale both knobs when tuning for throughput","Validate the pair when migrating legacy hnsw:batch_size / hnsw:sync_threshold values"],"tags":["chromadb","configuration","hnsw","batch-size","sync-threshold","invariant"],"backgroundTag":"configuration-validation-failed","analyzedSha":"aecdd12c8a891610db8653630b066b32ceb678b5","analyzedAt":"2026-08-16T21:53:27.228Z","schemaVersion":2},"datasetVersion":"2026-08-16T23:17:17.608Z"}