huggingface/transformers · error · Exception

Workspace must be set before calling forward

Error message

Workspace must be set before calling forward

What it means

`HiggsLinear` (Higgs quantized linear layer) relies on an external vLLM-style kernel (`qgemm_v2`) that needs a preallocated `workspace` buffer and `tune_metadata`, both intentionally left as None at construction because they are architecture/GPU dependent and meant to be shared across layers. Calling `forward()` before those attributes are populated raises this Exception.

Source

Thrown at src/transformers/integrations/higgs.py:536

        )
        self.tables = nn.Parameter(torch.empty((2**num_bits,), dtype=dtype, device=device), requires_grad=False)
        self.tables2 = nn.Parameter(
            torch.empty((2**num_bits, 2**num_bits, 2), dtype=dtype, device=device), requires_grad=False
        )

        if bias:
            self.bias = nn.Parameter(torch.empty(out_features, device=device, dtype=dtype), requires_grad=False)
        else:
            self.register_parameter("bias", None)

        self.workspace = None  # must be set externally to be reused among layers
        self.tune_metadata: TuneMetaData = None  # must be set externally because architecture dependent

    def forward(self, x):
        x = pad_to_block(x, [-1], self.hadamard_size)

        if self.workspace is None:
            raise Exception("Workspace must be set before calling forward")

        return qgemm_v2(
            x,
            self.weight,
            self.scales,
            self.tables,
            self.tables2.view(dtype=torch.float32),
            self.workspace,
            self.tune_metadata,
            hadamard_size=self.hadamard_size,
        )


def replace_with_higgs_linear(model, modules_to_not_convert: list[str] | None = None, quantization_config=None):
    """
    Public method that replaces the Linear layers of the given model with HIGGS quantized layers.

    Args:

View on GitHub (pinned to a597f97485)

Solutions

1. Run the kernel tuning step for your GPU and assign the results before the first forward: set `layer.workspace` and `layer.tune_metadata` (see vLLM's `tune`/`qgemm` workspace APIs the integration is modeled on).
2. Share one workspace across all HiggsLinear layers (it is deliberately external for reuse) but tune once per architecture.
3. For quick tests, verify wiring with `layer.weight`/dequantized matmul instead of calling `forward()` untuned.

Example fix

# before
lin = HiggsLinear(in_features, out_features, true_seqlen=True)
y = lin(x)  # Exception: Workspace must be set before calling forward

# after (vLLM-style tuning)
from vllm import _custom_ops as ops
lin.workspace, lin.tune_metadata = ops.tune(
    lin.weight, lin.tables, lin.tables2, lin.scales, 1.0, 10, lin.hadamard_size
)
y = lin(x)

assert lin.workspace is not None and lin.tune_metadata is not None, (
    "run kernel tuning and assign workspace/tune_metadata before forward"
)

def higgs_linear_ready(lin) -> bool:
    return lin.workspace is not None and lin.tune_metadata is not None

try:
    y = lin(x)
except Exception as e:
    if "Workspace must be set" not in str(e):
        raise
    tune_and_assign(lin)  # run kernel tuning, set workspace/tune_metadata
    y = lin(x)

When it happens

Trigger: Instantiating `HiggsLinear` and immediately running `layer(x)` without first assigning `layer.workspace = torch.empty(...)` and `layer.tune_metadata = ...` (typically obtained via the kernel's tuning routine per GPU architecture), e.g. in a plain PyTorch training loop that never calls the vLLM/flashinfer tune step.

Common situations: Using Higgs quantization outside the intended vLLM-style inference stack where tuning is done up front; moving code to a new GPU and forgetting to re-run tuning; unit-testing the layer in isolation.

Related errors

• Unsupported p={p}, n={n}
• TP and DP cannot be used together
• You need to install optimum-quanto in order to use KV cache
• `nbits` for `quanto` backend has to be one of [`2`, `4`] but
• `axis_key` for `quanto` backend has to be one of [`0`, `-1`]

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/3fa63a4d6edcacea. Report an issue: GitHub.