sgl-project/sglang · error · ImportError

Humming quantization requires `humming-kernels`. Please inst

Error message

Humming quantization requires `humming-kernels`. Please install it to use `--quantization humming`.

What it means

The Humming (W4A8/FP8) quantization path lazily imports `humming-kernels`; if the `humming` package is missing (ModuleNotFoundError) or its internal imports fail, _lazy_import_humming raises ImportError telling you to install humming-kernels to use --quantization humming.

Source

Thrown at python/sglang/srt/layers/quantization/humming.py:80

        from humming.dtypes import DataType as _DataType
        from humming.layer import HummingMethod as _HummingMethod
        from humming.schema import BaseInputSchema as _BaseInputSchema
        from humming.schema import BaseWeightSchema as _BaseWeightSchema
        from humming.schema import HummingInputSchema as _HummingInputSchema
        from humming.schema import HummingWeightSchema as _HummingWeightSchema
        from humming.utils.weight import quantize_weight as _quantize_weight
    except ImportError as err:
        if isinstance(err, ModuleNotFoundError) and err.name == "humming":
            message = (
                "Humming quantization requires `humming-kernels`. "
                "Please install it to use `--quantization humming`."
            )
        else:
            message = (
                "Failed to import Humming quantization dependencies from "
                f"`humming-kernels`: {err}"
            )
        raise ImportError(message) from err

    DataType = _DataType
    HummingMethod = _HummingMethod
    BaseInputSchema = _BaseInputSchema
    BaseWeightSchema = _BaseWeightSchema
    HummingInputSchema = _HummingInputSchema
    HummingWeightSchema = _HummingWeightSchema
    quantize_weight = _quantize_weight


def prepare_padded_shape(shape, x):
    padded_shape = math.ceil(shape / x) * x
    return padded_shape, padded_shape - shape


def prepare_param(tensor, name, extra_attrs):
    extra_attrs = extra_attrs.copy()
    scale_type = extra_attrs.pop("scale_type", None)

View on GitHub (pinned to 0132848349)

Solutions

pip install humming-kernels (matching the version required by your sglang release)
If already installed, upgrade/downgrade it to a compatible version and retry
Verify with python -c "import humming.dtypes, humming.layer, humming.schema" that all submodules import cleanly
If not intending to use Humming, remove --quantization humming or use a checkpoint with a different quant method

Example fix

# before
python -m sglang.launch_server --model humming-model --quantization humming  # ImportError

# after
pip install humming-kernels
python -m sglang.launch_server --model humming-model --quantization humming

Defensive patterns

Strategy: try-catch

Validate before calling

try:
    import humming.dtypes, humming.layer, humming.schema  # noqa
    humming_available = True
except ImportError:
    humming_available = False
if args.quantization == "humming" and not humming_available:
    raise SystemExit("pip install humming-kernels before using --quantization humming")

Type guard

def humming_ready() -> bool:
    try:
        import humming.layer  # noqa: F401
        return True
    except ImportError:
        return False

Try / catch

try:
    from sglang.srt.layers.quantization.humming import HummingConfig
except ImportError as e:
    if "humming-kernels" in str(e):
        sys.exit("Install humming-kernels: pip install humming-kernels")
    raise

Prevention

Bake humming-kernels into Docker images used for humming models
Gate --quantization humming behind an import check at startup
Pin a compatible humming-kernels version in requirements

When it happens

Trigger: Starting SGLang with --quantization humming (or loading a model whose quant_method is humming) without the humming-kernels wheel installed, or with an incompatible/broken version of it.

Common situations: GPU image that doesn't bundle humming-kernels; version mismatch where humming is installed but its dtypes/layer/schema submodules fail to import; CI environments missing optional dependencies.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/72fcaf08a47a71ab. Report an issue: GitHub.