BerriAI/litellm · error · Exception

Quantization config must be one of 'scalar', 'binary' or 'pr

Error message

Quantization config must be one of 'scalar', 'binary' or 'product'

What it means

When the collection does not yet exist, the cache creates it with a quantization_config. Only three string values are accepted: 'scalar', 'binary' (or None, which defaults to binary), and 'product'. Any other string — or a dict the code does not recognize — falls through to this exception. Note the error message text omits that None is also accepted (None means default binary quantization).

Source

Thrown at litellm/caching/qdrant_semantic_cache.py:125

            quantization_params: dict[str, Any]
            if quantization_config is None or quantization_config == "binary":
                quantization_params = {
                    "binary": {
                        "always_ram": False,
                    }
                }
            elif quantization_config == "scalar":
                quantization_params = {
                    "scalar": {
                        "type": "int8",
                        "quantile": QDRANT_SCALAR_QUANTILE,
                        "always_ram": False,
                    }
                }
            elif quantization_config == "product":
                quantization_params = {"product": {"compression": "x16", "always_ram": False}}
            else:
                raise Exception("Quantization config must be one of 'scalar', 'binary' or 'product'")

            new_collection_status: Final = self.sync_client.put(
                url=f"{self.qdrant_api_base}/collections/{self.collection_name}",
                json={
                    "vectors": {"size": self.vector_size, "distance": "Cosine"},
                    "quantization_config": quantization_params,
                },
                headers=self.headers,
            )
            if new_collection_status.json()["result"]:
                collection_details = self.sync_client.get(
                    url=f"{self.qdrant_api_base}/collections/{self.collection_name}",
                    headers=self.headers,
                )
                self.collection_info = collection_details.json()
                print_verbose(f"New collection created.\nCollection details:{self.collection_info}")
                self._ensure_cache_key_payload_index()
            else:

View on GitHub (pinned to 6c2dcb801b)

Solutions

Use one of the supported values: quantization_config='scalar', 'binary', or 'product'
To get the default binary quantization, omit quantization_config (or pass None)

Example fix

# before
cache = QdrantSemanticCache(..., quantization_config='int8')

# after
cache = QdrantSemanticCache(..., quantization_config='scalar')  # int8 scalar quantization

Defensive patterns

Strategy: validation

Validate before calling

ALLOWED_QUANTIZATION = {'scalar', 'binary', 'product'}
q = cfg.get('quantization_config')
if q is not None and q not in ALLOWED_QUANTIZATION:
    raise ValueError(f'quantization_config must be one of {sorted(ALLOWED_QUANTIZATION)} or omitted')

Prevention

Document the three accepted values wherever cache configs are authored
Use an enum/ Literal type in your own config layer so invalid values never reach litellm

When it happens

Trigger: Passing quantization_config='int8', 'none', 'off', 'fp16', or an arbitrary dict to QdrantSemanticCache when the collection is being created for the first time; existing collections skip this branch entirely.

Common situations: Trying to disable quantization with quantization_config='none' instead of leaving it out; guessing a quantization name from Qdrant docs that doesn't match the three allowed aliases.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/d2d350774b77a0d6. Report an issue: GitHub.