{"record":{"id":"1ece80127fa9a4aa","repo":"keras-team/keras","slug":"quantization-mode-mode-doesn-t-work-well-with","errorCode":null,"errorMessage":"Quantization mode='{mode}' doesn't work well with compute_dtype='float16'. Consider loading model/layer with another dtype policy such as 'mixed_bfloat16' or 'mixed_float16' before calling `quantize()`.","messagePattern":"Quantization mode='(.+?)' doesn't work well with compute_dtype='float16'\\. Consider loading model/layer with another dtype policy such as 'mixed_bfloat16' or 'mixed_float16' before calling `quantize\\(\\)`\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"keras/src/layers/layer.py","lineNumber":1386,"sourceCode":"            raise ValueError(\n                \"Cannot quantize a layer that isn't yet built. \"\n                f\"Layer '{self.name}' (of type '{self.__class__.__name__}') \"\n                \"is not built yet.\"\n            )\n        if getattr(self, \"_is_quantized\", False):\n            raise ValueError(\n                f\"Layer '{self.name}' is already quantized with \"\n                f\"dtype_policy='{self.dtype_policy.name}'. \"\n                f\"Received: mode={mode}\"\n            )\n        if mode not in dtype_policies.QUANTIZATION_MODES:\n            raise ValueError(\n                \"Invalid quantization mode. \"\n                f\"Expected one of {dtype_policies.QUANTIZATION_MODES}. \"\n                f\"Received: mode={mode}\"\n            )\n        if mode == \"int8\" and compute_dtype == \"float16\":\n            raise ValueError(\n                f\"Quantization mode='{mode}' doesn't work well with \"\n                \"compute_dtype='float16'. Consider loading model/layer with \"\n                \"another dtype policy such as 'mixed_bfloat16' or \"\n                \"'mixed_float16' before calling `quantize()`.\"\n            )\n\n    def quantized_call(self, *args, **kwargs):\n        current_remat_mode = get_current_remat_mode()\n\n        if (\n            current_remat_mode != self._remat_mode\n            and current_remat_mode is not None\n        ):\n            warnings.warn(\n                f\"The RematScope at call time ({current_remat_mode}) differs \"\n                f\"the one set during layer initialization \"\n                f\"({self._remat_mode}). \"\n                f\"Restoring the correct rematerialization mode \"","sourceCodeStart":1368,"sourceCodeEnd":1404,"githubUrl":"https://github.com/keras-team/keras/blob/7a34a03db60bf60042242d6a556fc3be119046a5/keras/src/layers/layer.py#L1368-L1404","documentation":"int8 quantization with compute_dtype='float16' is rejected because int8 kernels dequantize to float32 on many backends; float16 compute degrades accuracy or fails. Keras suggests mixed_bfloat16/mixed_float16 policies instead.","triggerScenarios":"Setting model.dtype_policy = 'mixed_float16' (or policy with compute_dtype float16) then calling quantize('int8').","commonSituations":"Memory-saving pipelines that combine fp16 training with int8 PTQ; porting models trained in float16 to int8 serving.","solutions":["Use dtype_policy='mixed_bfloat16' or another policy whose compute dtype is float32 before quantize('int8')'","If fp16 compute is required, pick a quantization mode that supports it rather than int8"],"exampleFix":"# before\nmodel.dtype_policy = 'mixed_float16'\nmodel.quantize('int8')\n# after\nmodel.dtype_policy = 'mixed_bfloat16'\nmodel.quantize('int8')","handlingStrategy":"validation","validationCode":"assert not (mode == 'int8' and layer.dtype_policy.compute_dtype == 'float16')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Use mixed_bfloat16 instead of mixed_float16 before int8 quantization","Check compute_dtype of the policy before quantize()"],"tags":["keras","quantization","dtype","float16"],"backgroundTag":"quantization-config-invalid","analyzedSha":"7a34a03db60bf60042242d6a556fc3be119046a5","analyzedAt":"2026-08-25T21:25:25.994Z","schemaVersion":2},"datasetVersion":"2026-08-26T02:17:13.382Z"}