tracel-ai/burn · error
Can't store native sub-byte values
Error message
Can't store native sub-byte values
What it means
When a quantization scheme uses `QuantStore::Native` (one value per storage element), burn-cubecl cannot natively store sub-byte quantization formats (Q4F, Q4S, Q2F, Q2S, E2M1) because Rust/hardware buffer element types are at least one byte. `quantized_handles` panics to tell you such schemes must use `QuantStore::PackedU32` instead.
Source
Thrown at crates/burn-cubecl/src/tensor/quantization.rs:67
meta: self.meta.clone(),
device: self.device.clone(),
dtype: DType::I8,
qparams: None,
},
QuantValue::E4M3 | QuantValue::E5M2 => CubeTensor {
client: self.client.clone(),
handle: self.handle.clone(),
meta: self.meta.clone(),
device: self.device.clone(),
dtype: DType::U8,
qparams: None,
},
QuantValue::Q4F
| QuantValue::Q4S
| QuantValue::Q2F
| QuantValue::Q2S
| QuantValue::E2M1 => {
panic!("Can't store native sub-byte values")
}
},
QuantStore::PackedU32(packed_dim) => {
let packed_dim = self.rank() - packed_dim - 1;
let mut shape = self.shape();
shape[packed_dim] = shape[packed_dim].div_ceil(scheme.num_quants());
CubeTensor {
client: self.client.clone(),
handle: self.handle.clone(),
meta: Box::new(Metadata::new(shape, self.meta.strides.clone())),
device: self.device.clone(),
dtype: DType::U32,
qparams: None,
}
}
QuantStore::PackedNative(packed_dim) => match scheme.value {
QuantValue::E2M1 => {View on GitHub (pinned to d16f7ba2ed)
Solutions
- Configure the scheme with `QuantStore::PackedU32(position)` so sub-byte values are packed 4- or 8-per u32: e.g. `QuantScheme::default().with_value(QuantValue::Q4F).with_store(QuantStore::PackedU32(PackedParam::Lhs))`.
- Use a scheme constructor that sets a compatible store for the chosen value instead of mixing a Q8-native scheme with a sub-byte value.
- If native storage is required, switch to a byte-aligned value such as `QuantValue::Q8F` / `Q8S`.
Example fix
// before
let scheme = QuantScheme::default()
.with_value(QuantValue::Q4F)
.with_store(QuantStore::Native); // panic
// after
let scheme = QuantScheme::default()
.with_value(QuantValue::Q4F)
.with_store(QuantStore::PackedU32(PackedParam::Lhs)); // sub-byte values packed Defensive patterns
Strategy: validation
Validate before calling
// validate scheme before quantizing let sub_byte = matches!(scheme.value, QuantValue::Q4F | QuantValue::Q4S | QuantValue::Q2F | QuantValue::Q2S | QuantValue::E2M1); assert!(!sub_byte || matches!(scheme.store, QuantStore::PackedU32(_)), "sub-byte QuantValue requires QuantStore::PackedU32");
Try / catch
// panic-based; fix the scheme construction rather than catching let scheme = scheme.with_store(QuantStore::PackedU32(PackedParam::Lhs));
Prevention
- Never build a QuantScheme by only swapping QuantValue on an existing scheme; set store mode together
- Use library-provided scheme constructors/constants for standard formats (e.g. Q4F, Q8S)
- Remember rule: native store only for byte-aligned values; sub-byte values must be packed in u32
- Add a unit test quantizing/dequantizing a tiny tensor with your exact scheme before training
When it happens
Trigger: Calling `tensor.quantize(&scheme)` / `into_contiguous_quantized`, `launch_matmul` with quantized inputs, `dequantize`, or `q_reshape` where the scheme is `QuantScheme` with `store = QuantStore::Native` and `value = QuantValue::Q4F | Q4S | Q2F | Q2S | E2M1`.
Common situations: Constructing a `QuantScheme` manually (e.g. `QuantScheme::default().with_value(QuantValue::Q4F)`) without also setting `.with_store(QuantStore::PackedU32(...))`; copying a scheme used for a native byte-size type (Q8) and swapping only the value to a 4-bit/2-bit format; upgrading library versions where defaults changed.
Related errors
- {other:?} doesn't support native packing
- Both tensors should be on the same device {:?} != {:?}
- Expected quantized dtype, got {:?}
- todo!("Quantization not supported yet")
- lookup quantization does not travel as a QFloat tensor
AI-assisted analysis of tracel-ai/burn@d16f7ba2ed (2026-09-05).
Data as JSON: /api/errors/296ce5fddba3f12d.
Report an issue: GitHub.