huggingface/transformers · error · ValueError

Unsupported config file format: {args.config_file}

Error message

Unsupported config file format: {args.config_file}

What it means

`deepgemm_fp8_fp4_linear` rejects `activation_scale is not None`. DeepGEMM performs per-token (dynamic, per-row) activation quantization via `per_token_cast_to_fp8`; a static per-tensor activation scale (from `activation_scheme='static'` checkpoints) is fundamentally incompatible with its kernels, so it raises `NotImplementedError` and expects callers to route static activations through the Triton fallback.

Source

Thrown at benchmark_v2/run_benchmarks.py:109

    if any(arg is None for arg in [args.batch_size, args.sequence_length, args.num_tokens_to_generate]):
        raise ValueError(
            "All of the arguments --batch-size, --sequence-length, and --num-tokens-to-generate are required"
        )

    # We cannot compute ITL if we don't have at least two measurements
    if any(n <= 1 for n in args.num_tokens_to_generate):
        raise ValueError("--num_tokens_to_generate arguments should be larger than 1")

    # If a config file is provided, read it and use the configs therein. They will still be adapted to the given arguments.
    if args.config_file is not None:
        if args.config_file.endswith(".json"):
            with open(args.config_file, "r") as f:
                config_as_dicts = [json.load(f)]
        elif args.config_file.endswith(".jsonl"):
            with open(args.config_file, "r") as f:
                config_as_dicts = [json.loads(line) for line in f if line.startswith("{")]
        else:
            raise ValueError(f"Unsupported config file format: {args.config_file}")
        configs = [BenchmarkConfig.from_dict(config) for config in config_as_dicts]
    else:
        # Otherwise, get the configs for the given coverage level
        configs = get_config_by_level(args.level)

    # Adapt the configs to the given arguments
    configs = adapt_configs(
        configs,
        args.warmup,
        args.iterations,
        args.batch_size,
        args.sequence_length,
        args.num_tokens_to_generate,
        not args.no_gpu_monitoring,
    )

    if args.enable_tp:
        for config in configs:

View on GitHub (pinned to a597f97485)

Solutions

1. Route static-activation models through the Triton FP8 linear instead of DeepGEMM (the intended fallback)
2. Re-quantize with `activation_scheme="dynamic"` so no per-tensor activation scale exists
3. In integration code, gate on `activation_scheme == "static"` before choosing the DeepGEMM path

Example fix

# before
output = deepgemm_fp8_fp4_linear(x, w, w_scale, activation_scale=static_scale)

# after
if activation_scale is not None:
    output = fp8_linear_torch(x, w, w_scale, activation_scale)   # Triton/default path
else:
    output = deepgemm_fp8_fp4_linear(x, w, w_scale)

if activation_scale is not None:  # static per-tensor scheme
    output = fp8_linear_triton(x, w, w_sf, activation_scale)
else:
    output = deepgemm_fp8_fp4_linear(x, w, w_sf)

try:
    out = deepgemm_fp8_fp4_linear(x, w, w_sf, activation_scale=scale)
except NotImplementedError:
    out = fp8_linear_triton(x, w, w_sf, activation_scale)

When it happens

Trigger: Calling the DeepGEMM linear with a checkpoint quantized under `activation_scheme="static"` (calibrated per-tensor input scales), where the caller passes `activation_scale` into `deepgemm_fp8_fp4_linear`.

Common situations: DeepSeek-V2-style static-FP8 checkpoints (calibrated `input_scale`) run with the DeepGEMM linear; generic integration code that always forwards `activation_scale` when present.

Related errors

• Incorrect audio source. Must be a valid URL starting with `h
• No benchmark was run successfully
• All of the arguments --batch-size, --sequence-length, and --
• --num_tokens_to_generate arguments should be larger than 1
• min should be < max (got min: {min}, max: {max})

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/f65fdc751e1a5e4c. Report an issue: GitHub.