keras-team/keras · error · ValueError

Quantization mode='{mode}' doesn't work well with compute_dt

Error message

Quantization mode='{mode}' doesn't work well with compute_dtype='float16'. Consider loading model/layer with another dtype policy such as 'mixed_bfloat16' or 'mixed_float16' before calling `quantize()`.

What it means

int8 quantization with compute_dtype='float16' is rejected because int8 kernels dequantize to float32 on many backends; float16 compute degrades accuracy or fails. Keras suggests mixed_bfloat16/mixed_float16 policies instead.

Source

Thrown at keras/src/layers/layer.py:1386

            raise ValueError(
                "Cannot quantize a layer that isn't yet built. "
                f"Layer '{self.name}' (of type '{self.__class__.__name__}') "
                "is not built yet."
            )
        if getattr(self, "_is_quantized", False):
            raise ValueError(
                f"Layer '{self.name}' is already quantized with "
                f"dtype_policy='{self.dtype_policy.name}'. "
                f"Received: mode={mode}"
            )
        if mode not in dtype_policies.QUANTIZATION_MODES:
            raise ValueError(
                "Invalid quantization mode. "
                f"Expected one of {dtype_policies.QUANTIZATION_MODES}. "
                f"Received: mode={mode}"
            )
        if mode == "int8" and compute_dtype == "float16":
            raise ValueError(
                f"Quantization mode='{mode}' doesn't work well with "
                "compute_dtype='float16'. Consider loading model/layer with "
                "another dtype policy such as 'mixed_bfloat16' or "
                "'mixed_float16' before calling `quantize()`."
            )

    def quantized_call(self, *args, **kwargs):
        current_remat_mode = get_current_remat_mode()

        if (
            current_remat_mode != self._remat_mode
            and current_remat_mode is not None
        ):
            warnings.warn(
                f"The RematScope at call time ({current_remat_mode}) differs "
                f"the one set during layer initialization "
                f"({self._remat_mode}). "
                f"Restoring the correct rematerialization mode "

View on GitHub (pinned to 7a34a03db6)

Solutions

Use dtype_policy='mixed_bfloat16' or another policy whose compute dtype is float32 before quantize('int8')'
If fp16 compute is required, pick a quantization mode that supports it rather than int8

Example fix

# before
model.dtype_policy = 'mixed_float16'
model.quantize('int8')
# after
model.dtype_policy = 'mixed_bfloat16'
model.quantize('int8')

Defensive patterns

Strategy: validation

Validate before calling

assert not (mode == 'int8' and layer.dtype_policy.compute_dtype == 'float16')

Prevention

Use mixed_bfloat16 instead of mixed_float16 before int8 quantization
Check compute_dtype of the policy before quantize()

When it happens

Trigger: Setting model.dtype_policy = 'mixed_float16' (or policy with compute_dtype float16) then calling quantize('int8').

Common situations: Memory-saving pipelines that combine fp16 training with int8 PTQ; porting models trained in float16 to int8 serving.

Related errors

AI-assisted analysis of keras-team/keras@7a34a03db6 (2026-08-25). Data as JSON: /api/errors/1ece80127fa9a4aa. Report an issue: GitHub.