keras-team/keras · error · ValueError

Implicitly enabling GPTQ quantization by setting `dtype_poli

Error message

Implicitly enabling GPTQ quantization by setting `dtype_policy` to '{value}' is not supported. GPTQ requires a calibration dataset and a `GPTQConfig` object.

Please use the `.quantize('gptq', config=...)` method on the layer or model instead.

What it means

GPTQ quantization needs a calibration dataset and a GPTQConfig, so Keras refuses to enable it implicitly when you assign a dtype_policy string like 'gptq' to an already-built layer. Other quantization modes (int8, float8) can be enabled via dtype_policy, but 'gptq' cannot.

Source

Thrown at keras/src/layers/layer.py:809

            variable.assign(value)

    @property
    def dtype_policy(self):
        return self._dtype_policy

    @dtype_policy.setter
    def dtype_policy(self, value):
        policy = dtype_policies.get(value)
        if isinstance(self._dtype_policy, DTypePolicyMap) and self.path:
            if self.path in self._dtype_policy:
                del self._dtype_policy[self.path]
            self._dtype_policy[self.path] = policy
        else:
            self._dtype_policy = policy
        if policy.quantization_mode is not None:
            if self.built and not getattr(self, "_is_quantized", False):
                if policy.quantization_mode == "gptq":
                    raise ValueError(
                        "Implicitly enabling GPTQ quantization by setting "
                        f"`dtype_policy` to '{value}' is not supported. "
                        "GPTQ requires a calibration dataset and a "
                        "`GPTQConfig` object.\n\n"
                        "Please use the `.quantize('gptq', config=...)` method "
                        "on the layer or model instead."
                    )
                self.quantize(policy.quantization_mode)

    @property
    def dtype(self):
        """Alias of `layer.variable_dtype`."""
        return self.variable_dtype

    @property
    def compute_dtype(self):
        """The dtype of the computations performed by the layer."""
        if isinstance(self._dtype_policy, DTypePolicyMap) and self.path:

View on GitHub (pinned to 7a34a03db6)

Solutions

1. Call layer.quantize('gptq', config=GPTQConfig(...)) or model.quantize('gptq', config=...) with calibration data instead of setting dtype_policy
2. If you only need weight-only int8, use a non-GPTQ policy such as 'int8' via dtype_policy

Example fix

# before
layer.dtype_policy = 'int8_gptq'
# after
from keras.quantizers import GPTQConfig
model.quantize('gptq', config=GPTQConfig(bits=4, calibration_data=calib))

if layer.dtype_policy.quantization_mode == 'gptq':
    raise SystemExit('enable GPTQ via quantize(), not dtype_policy')

When it happens

Trigger: Setting layer.dtype_policy = 'int8_gptq' (or a policy whose quantization_mode == 'gptq') on a built layer, or deserializing such a policy without going through quantize().

Common situations: Copying dtype policy strings from int8 examples and swapping in gptq; trying to reproduce a quantized checkpoint by only setting the policy.

Related errors

• lora is not currently supported with GPTQ quantization.
• Cannot save layer '{self.name}' because it is quantized with
• lora is not currently supported with GPTQ quantization.
• Cannot save layer '{self.name}' because it is quantized with
• Could not determine row/column split.

AI-assisted analysis of keras-team/keras@7a34a03db6 (2026-08-25). Data as JSON: /api/errors/661a45d2282d756f. Report an issue: GitHub.