karpathy/nanochat · error · ValueError

Only 'tensorwise' recipe is supported, got '{recipe_name}'.

Error message

Only 'tensorwise' recipe is supported, got '{recipe_name}'. Rowwise/axiswise recipes require the full torchao library.

What it means

nanochat/fp8.py ships a minimal, vendored subset of torchao's float8 training API. `Float8LinearConfig.from_recipe_name` accepts only the 'tensorwise' recipe (cast the whole tensor with a single scaling factor). Rowwise/axiswise per-channel recipes are intentionally unsupported because they require the full torchao library, so requesting one raises ValueError.

Source

Thrown at nanochat/fp8.py:236

        Uses meta device to avoid allocating a temporary weight tensor — we
        create the module shell on meta (shapes/dtypes only, no memory), then
        point .weight and .bias to the original module's parameters.
        """
        with torch.device("meta"):
            new_mod = cls(mod.in_features, mod.out_features, bias=False)
        new_mod.weight = mod.weight
        new_mod.bias = mod.bias
        return new_mod


class Float8LinearConfig:
    """Minimal config matching torchao's API. Only tensorwise recipe is supported."""

    @staticmethod
    def from_recipe_name(recipe_name):
        if recipe_name != "tensorwise":
            raise ValueError(
                f"Only 'tensorwise' recipe is supported, got '{recipe_name}'. "
                f"Rowwise/axiswise recipes require the full torchao library."
            )
        return Float8LinearConfig()


def convert_to_float8_training(module, *, config=None, module_filter_fn=None):
    """Replace nn.Linear layers with Float8Linear throughout a module.

    Walks the module tree in post-order (children before parents) and swaps
    each nn.Linear that passes the optional filter. The new Float8Linear shares
    the original weight and bias tensors — no copies, no extra memory.

    Args:
        module: Root module to convert.
        config: Float8LinearConfig (accepted for API compat, only tensorwise supported).
        module_filter_fn: Optional filter(module, fqn) -> bool. Only matching Linears
            are converted. Common use: skip layers with dims not divisible by 16

View on GitHub (pinned to 92d63d4e8b)

Solutions

1. Use recipe_name='tensorwise' (or omit it) with nanochat's built-in fp8 path.
2. If you truly need rowwise/axiswise scaling, install and use the full torchao library (torchao.float8) instead of nanochat/fp8.py.
3. Check the training script's fp8 recipe argument and change its default.

Example fix

# before
cfg = Float8LinearConfig.from_recipe_name("rowwise")

# after
cfg = Float8LinearConfig.from_recipe_name("tensorwise")

recipe = "tensorwise"  # nanochat/fp8.py supports only this
assert recipe == "tensorwise", "use the full torchao library for rowwise/axiswise"

def is_supported_fp8_recipe(name: str) -> bool:
    return name == "tensorwise"

When it happens

Trigger: Calling `Float8LinearConfig.from_recipe_name('rowwise')` or `'axiswise'` (or any non-'tensorwise' string) — typically via a training script flag like --fp8 recipe that is passed through to this factory.

Common situations: Copying a torchao float8 config from another project that used 'rowwise' and running it under nanochat's built-in fp8 path; upgrading scripts expecting full torchao recipe support.

Related errors

• Unknown dataset tag: {dataset_tag}
• Unknown optimizer kind: {group['kind']}
• No training horizon specified
• No checkpoints found in {checkpoints_dir}
• No checkpoints found in {checkpoint_dir}

AI-assisted analysis of karpathy/nanochat@92d63d4e8b (2026-08-15). Data as JSON: /api/errors/2272f66e725658f2. Report an issue: GitHub.