tracel-ai/burn · error

Not yet supported

Error message

Not yet supported

What it means

QuantizedBytes::split_values_off (crates/burn-std/src/tensor/quantization.rs:313-348) splits quantized values from quantization parameters. For PackedU32 storage it handles Q8/Q4/Q2 values (unpacking sub-byte data to i8), but for float-point values E4M3/E5M2/E2M1 in PackedU32 storage, and for any PackedNative storage, it panics with `unimplemented!("Not yet supported")` — packed float-point-quantized data cannot yet be split/unpacked.

Source

Thrown at crates/burn-std/src/tensor/quantization.rs:341

        }

        let (values, qparams) = match self.scheme.store {
            QuantStore::Native => self.split_i8_values(scale_bytes),
            QuantStore::PackedU32(_) => match self.scheme.value {
                QuantValue::Q8F | QuantValue::Q8S => self.split_i8_values(scale_bytes),
                QuantValue::Q4F | QuantValue::Q4S | QuantValue::Q2F | QuantValue::Q2S => {
                    let split_at =
                        self.bytes.len().checked_sub(scale_bytes).expect(
                            "quantized tensor data is shorter than its scheme's parameters",
                        );
                    let qparams = self.bytes[split_at..].to_vec();
                    let values = bytemuck::cast_slice::<_, u32>(&self.bytes[..split_at]);
                    // Sub-byte values are unpacked as i8s for value equality tests
                    let values = unpack_q_to_i8s(values, self.num_elements(), &self.scheme.value);
                    (values, qparams)
                }
                QuantValue::E4M3 | QuantValue::E5M2 | QuantValue::E2M1 => {
                    unimplemented!("Not yet supported")
                }
            },
            QuantStore::PackedNative(_) => unimplemented!("Not yet supported"),
        };

        (values, (qparams, num_params))
    }
}

/// Round a scale up to the smallest value representable by the scale dtype that is no smaller.
///
/// Backends that keep scales in `f32` must apply this when quantizing, so that the scale they
/// divide by is the one that will actually be stored. Otherwise a tensor dequantizes differently
/// after a save/load round trip.
///
/// Up rather than to nearest, because a scale is derived from the largest magnitude it has to
/// cover. Rounding down puts that value past the end of the quantized range, where it clips, which
/// measured several times worse than the coarser step rounding up costs.

View on GitHub (pinned to d16f7ba2ed)

Solutions

1. Avoid packed storage for E4M3/E5M2/E2M1 schemes: use QuantStore::Native (byte-aligned float-point values) so split_i8_values runs instead.
2. Dequantize via a supported path (native-store dequantize) instead of into_vec_i8 for packed float-point data.
3. If packing is not needed, construct the scheme with no packed dimension (packed_dim 0 / native store).
4. Track upstream burn for packed float-point quantization support and upgrade.

Example fix

// before
let scheme = QuantScheme::default().with_value(QuantValue::E4M3); // may pack E4M3 -> PackedU32
let (vals, params) = q_bytes.into_vec_i8(); // panics
// after
let f_data = quantized.dequantize(); // supported float-point path
let vals = f_data.iter::<f32>().collect::<Vec<_>>();

fn can_split_values(scheme: &QuantScheme) -> bool {
    match scheme.store {
        QuantStore::Native => true,
        QuantStore::PackedU32(_) => !matches!(
            scheme.value,
            QuantValue::E4M3 | QuantValue::E5M2 | QuantValue::E2M1
        ),
        QuantStore::PackedNative(_) => false,
    }
}

fn split_supported(store: &QuantStore) -> bool {
    !matches!(store, QuantStore::PackedNative(_))
}

// into_vec_i8 panics on unsupported stores; check first
if can_split_values(&scheme) {
    let (vals, params) = q_bytes.into_vec_i8();
} else {
    let floats = dequantize_native(&q_bytes); // alternate supported path
}

When it happens

Trigger: Calling into_vec_i8() (which calls split_values_off) on a QuantizedBytes with scheme.store == QuantStore::PackedU32 and scheme.value of E4M3/E5M2/E2M1, or with store == QuantStore::PackedNative (any value).

Common situations: Comparing or converting packed FP8/FP4 quantized tensor data (e.g. deserialized from a checkpoint saved with packing enabled) where the code path needs values as i8/floats.

Related errors

• todo!("Quantization not supported yet")
• unimplemented!()
• Can't format yet
• Not yet implemented for iteration
• lookup quantization is not supported for iteration

AI-assisted analysis of tracel-ai/burn@d16f7ba2ed (2026-09-05). Data as JSON: /api/errors/eb9a2b6153cbb8e4. Report an issue: GitHub.