huggingface/transformers · error · RuntimeError

No benchmark was run successfully

Error message

No benchmark was run successfully

What it means

`_assert_sm100_requirements` on a Blackwell GPU (SM100+) saw a float32 scale-factor tensor. DeepGEMM on SM100 only consumes UE8M0 scale factors; a checkpoint quantized with `quantization_config.scale_fmt='float'` carries plain float32 block scales, and rounding them to UE8M0 would silently corrupt outputs, so the guard raises `NotImplementedError` (treated by `fp8_linear` as a decline → Triton fallback).

Source

Thrown at benchmark_v2/framework/benchmark_runner.py:389

            # Memoize
            all_results[config.hash] = {
                "metadata": BenchmarkMetadata(
                    model_id=model_id,
                    branch_name=self.branch_name,
                    commit_id=self.commit_id,
                    commit_message=self.commit_message,
                    success=result is not None,
                ),
                "measurements": result if result is not None else BenchmarkResult(),
                "config": config,
            }

            # Cleanup model and save results
            self.cleanup()
            self.save_results(model_id, all_results, timestamp=timestamp, summarized=summarized)

        if len(all_results) < 1:
            raise RuntimeError("No benchmark was run successfully")

        if pretty_print_summary:
            if not self._is_primary_process():
                return (timestamp, all_results)
            print()
            print("=" * 100)
            print(f"Finished benchmarks in {time.perf_counter() - start_time:.2f} seconds")
            print(f"Total number of benchmarks: {len(all_results)}")
            print("First run metadata:")
            first_key = list(all_results.keys())[0]
            first_metadata = all_results[first_key]["metadata"].to_dict()
            hardware_info = first_metadata.pop("hardware_info")
            pretty_print_dict(first_metadata | hardware_info, tabs=1)
            for result in all_results.values():
                print("=" * 100)
                print(f"Config: {result['config'].infer_name(compact=False)}\n")
                result["measurements"].pprint(
                    batch_size=result["config"].batch_size,

View on GitHub (pinned to a597f97485)

Solutions

1. Use a checkpoint quantized with `scale_fmt='ue8m0'` on Blackwell GPUs
2. Or keep the float-scale checkpoint and let FP8 linear fall back to Triton (automatic; ensure NotImplementedError is not swallowed as fatal)
3. For expert layers, switch to `model.set_experts_implementation('grouped_mm')` which consumes float32 block scales directly

Example fix

// before
# DeepSeek-V3 float-SF checkpoint on B200 via DeepGEMM -> NotImplementedError
model.set_experts_implementation("deepgemm")

// after
model = AutoModelForCausalLM.from_pretrained("<ue8m0-quantized-model>")
# or
model.set_experts_implementation("grouped_mm")

from transformers.integrations.deepgemm import is_sm100

scale_fmt = getattr(model.config.quantization_config, "scale_fmt", "float")
if is_sm100() and scale_fmt == "float":
    model.set_experts_implementation("grouped_mm")  # float32 SFs unsupported on SM100

try:
    out = deepgemm_fp8_fp4_linear(x, w, w_sf)
except NotImplementedError:
    out = triton_fp8_linear(x, w, w_sf)  # consumes float32 block scales directly

When it happens

Trigger: Running a DeepSeek-style FP8 checkpoint quantized with `scale_fmt='float'` (float32 scales, e.g. DeepSeek-V3 float-SF variants) on B200/B300 via the DeepGEMM linear; or experts path with float32 `weight_scale_inv` on SM100.

Common situations: FP8 checkpoints published with float scales (DSv3 style) run on new Blackwell nodes; teams migrating Hopper pipelines to SM100 assuming the same checkpoint works; mixing UE8M0 and float scale checkpoints in one fleet.

Related errors

• Generated {results.size(-1)} tokens, expected {config.num_to
• All of the arguments --batch-size, --sequence-length, and --
• --num_tokens_to_generate arguments should be larger than 1
• Unsupported config file format: {args.config_file}
• Incorrect audio source. Must be a valid URL starting with `h

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/f850af1170b19f42. Report an issue: GitHub.