huggingface/transformers · error · ValueError

PUSH_TO_HUB_TOKEN is not set, cannot push results to the Hub

Error message

PUSH_TO_HUB_TOKEN is not set, cannot push results to the Hub. When setting dataset_id, please also set the PUSH_TO_HUB_TOKEN environment variable.

What it means

In the scale-factor layout coercion helper (`_coerce_sf_for_kernel`-style path), after dtype normalization the tensor `sf.dim()` is neither 2 nor 3, which is the only rank DeepGEMM kernels accept for activation/weight scale factors. This is a programming/shape error in how the caller prepared the SF tensor (e.g. a per-tensor scalar, or a 4D batched tensor).

Source

Thrown at benchmark_v2/framework/benchmark_runner.py:447

        # Convert results to dict
        converted_results = {}
        for cfg_hash in results.keys():
            converted_results[cfg_hash] = {
                "metadata": results[cfg_hash]["metadata"].to_dict(),
                "measurements": results[cfg_hash]["measurements"].to_dict(summarized=summarized),
                "config": results[cfg_hash]["config"].to_dict(),
            }

        # Save to JSON file
        with open(filepath, "w") as f:
            f.write(compact_json_numeric_arrays(converted_results))

        self.logger.info(f"Results saved to {filepath}")
        return filepath

    def push_results_to_hub(self, dataset_id: str, results: dict[Any, Any], timestamp: str) -> None:
        if PUSH_TO_HUB_TOKEN is None:
            raise ValueError(
                "PUSH_TO_HUB_TOKEN is not set, cannot push results to the Hub. When setting dataset_id, please also set the PUSH_TO_HUB_TOKEN environment variable."
            )

        api = HfApi()
        n_results = len(results)
        for summarized in [False, True]:
            self.logger.info(f"Pushing {n_results} results to: {dataset_id} with {summarized = }")
            rows = []
            for cfg_hash, entry in results.items():
                row = {
                    "benchmark_config_hash": cfg_hash,
                    "config": entry["config"].to_dict(),
                    "measurements": entry["measurements"].to_dict(summarized=summarized),
                    "metadata": entry["metadata"].to_dict(),
                }
                rows.append(row)

            ds = Dataset.from_list(rows)

View on GitHub (pinned to a597f97485)

Solutions

Inspect `sf.shape` right before the call and reshape to 2D `(rows, cols)` or 3D `(groups, rows, cols)` block-scale layout
If the scale is per-tensor, re-quantize block-wise (per 128-block) since DeepGEMM needs block SFs
Squeeze/remove spurious singleton dims from your scale pipeline (`.squeeze(-1)`, fix the `view` in your preprocessing)

Example fix

# before
sf = scales.reshape(1, -1)          # dim()==1 -> ValueError
out = deepgemm_fp8_linear(x, w, sf)

# after
sf = scales.reshape(rows, cols)      # 2D block SF layout
out = deepgemm_fp8_linear(x, w, sf)

Defensive patterns

Strategy: validation

Validate before calling

def check_sf_rank(sf: torch.Tensor) -> None:
    if sf.dim() not in (2, 3):
        raise ValueError(f"bad SF rank {sf.dim()}; expected 2D (rows, cols) or 3D (groups, rows, cols)")

check_sf_rank(scale_2d)

Type guard

def is_valid_sf(t: torch.Tensor) -> bool:
    """DeepGEMM scale factors must be rank 2 or 3."""
    return isinstance(t, torch.Tensor) and t.dim() in (2, 3)

Try / catch

try:
    out = deepgemm_fp8_fp4_linear(x, w, sf)
except ValueError as e:
    if "SF must be 2D or 3D" in str(e):
        sf = sf.reshape(-1, sf.size(-1))  # or fix the producer
        out = deepgemm_fp8_fp4_linear(x, w, sf)
    else:
        raise

Prevention

Assert scale tensor rank right after producing it in your quantization pipeline
Keep SF layout conventions written down (rows=K/128 blocks, cols=N or vice versa)
Add unit checks for scale shapes in preprocessing scripts

When it happens

Trigger: Passing an SF tensor with rank 1 (single flat scale vector), rank 0 (scalar per-tensor scale), or rank 4+ into a DeepGEMM linear/experts forward; commonly from custom quantization code that produced scales in an unexpected layout (e.g. `(E, N/128, K/128, 1)` kept with a trailing singleton instead of squeezing).

Common situations: Custom FP8 wrappers building their own scale tensors; converting HF FBGEMM-style scales `(1, K//128)` without reshape; offline preprocessing scripts that save scales with an extra batch dim.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/57964594f063d95f. Report an issue: GitHub.