fishaudio/fish-speech · error · ValueError

Invalid quantization mode {mode} needs to be one of [int8, i

Error message

Invalid quantization mode {mode} needs to be one of [int8, int4, int4-gpptq]

What it means

tools/llama/quantize.py supports only the modes int8, int4, and int4-gpptq; any other --mode value falls through to ValueError before weights are written.

Source

Thrown at tools/llama/quantize.py:486

            (dst_name / vq_model).unlink()
        quantize_path = dst_name / "model.pth"

    elif mode == "int4":
        print(
            "Quantizing model weights for int4 weight-only affine per-channel groupwise quantization"
        )
        quant_handler = WeightOnlyInt4QuantHandler(model, groupsize)
        quantized_state_dict = quant_handler.create_quantized_state_dict()

        dir_name = checkpoint_path
        dst_name = Path(f"checkpoints/fs-1.2-int4-g{groupsize}-{now}")
        shutil.copytree(str(dir_name.resolve()), str(dst_name.resolve()))
        if (dst_name / vq_model).exists():
            (dst_name / vq_model).unlink()
        quantize_path = dst_name / "model.pth"

    else:
        raise ValueError(
            f"Invalid quantization mode {mode} needs to be one of [int8, int4, int4-gpptq]"
        )

    print(f"Writing quantized weights to {quantize_path}")
    quantize_path.unlink(missing_ok=True)  # remove existing file if one already there
    torch.save(quantized_state_dict, quantize_path)
    print(f"Quantization complete took {time.time() - t0:.02f} seconds")


if __name__ == "__main__":
    quantize()

View on GitHub (pinned to befe400174)

Solutions

1. Use one of: int8, int4, int4-gpptq
2. Check `python tools/llama/quantize.py --help` in your checkout for the currently supported list
3. Update fish-speech if you need a mode added in newer versions

Example fix

# before
python tools/llama/quantize.py ... --mode int4_gptq
# after
python tools/llama/quantize.py ... --mode int4-gpptq

assert mode in {"int8", "int4", "int4-gpptq"}, f"unsupported mode: {mode}"

When it happens

Trigger: Running quantize.py --mode int8x or --mode q4f16 etc. — any string outside the supported set.

Common situations: Typos, or assuming a quantization format from other tools (GGUF, AWQ) is supported here; outdated docs naming different modes.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Number of prompt text ({len(prompt_text)}) and prompt audio
• Directory {path} does not exist.
• --reference_audio and --reference_text must be given the sam
• Reference audio file not found: {ref_audio}

AI-assisted analysis of fishaudio/fish-speech@befe400174 (2026-08-27). Data as JSON: /api/errors/b93f6754d15d214a. Report an issue: GitHub.