headroomlabs-ai/headroom · error · KompressModelNotCached

answerdotai/ModernBERT-base

Error message

answerdotai/ModernBERT-base

What it means

This is KompressModelNotCached("answerdotai/ModernBERT-base") raised while loading the tokenizer: the shim tries transformers' auto_tokenizer.from_pretrained(..., local_files_only=True), catches the not-cached error class, and when allow_download=False refuses to hit the network, raising this RuntimeError instead. The tokenizer is hardcoded to the ModernBERT-base repo because it pairs with the Kompress ONNX/PyTorch artifacts.

Source

Thrown at headroom/transforms/kompress_compressor.py:767

    """Load the ModernBERT tokenizer, cache-only when ``allow_download`` is False.

    Always tries the local cache FIRST, even when downloading is allowed. With
    ``local_files_only=False`` transformers re-validates against the Hub on every
    load — a tree listing plus a HEAD per tokenizer file — even when the repo is
    fully cached. MEASURED ~900ms warm-cache versus ~150ms local-only, i.e. ~750ms
    of pure network round-trip on every process start, and it is also what makes
    a cold start slow on a bad network rather than merely offline.

    Same files, same tokenizer, so the loaded object is identical; this only
    changes whether the Hub is consulted to confirm what is already on disk.
    Mirrors ``onnx_runtime.hf_hub_download_local_first``, which the ONNX half of
    this loader already uses.
    """
    try:
        return auto_tokenizer.from_pretrained("answerdotai/ModernBERT-base", local_files_only=True)
    except _NOT_CACHED_ERRORS as exc:
        if not allow_download:
            raise KompressModelNotCached("answerdotai/ModernBERT-base") from exc
    # Genuine cache miss and downloading is permitted: fetch it.
    return auto_tokenizer.from_pretrained("answerdotai/ModernBERT-base", local_files_only=False)


# Sub-state-dict keys inside a merged v2-style checkpoint (see
# scripts/export_kompress_v2_onnx.py, which this mirrors).
_MERGED_CHECKPOINT_KEYS = ("encoder_state_dict", "token_head_state_dict", "span_conv_state_dict")


def _load_merged_state_dict(model: Any, ckpt_path: str, model_id: str) -> None:
    """Load a merged v2-style checkpoint (LoRA already folded into the encoder).

    The checkpoint is a dict of per-submodule state-dicts
    (``encoder_state_dict`` / ``token_head_state_dict`` / ``span_conv_state_dict``)
    rather than a single flat state-dict, so each piece is loaded into its
    matching submodule directly instead of via a single ``load_state_dict``
    call on the whole model.
    """

View on GitHub (pinned to 322425c43b)

Solutions

1. Warm the tokenizer cache once with network allowed: huggingface-cli download answerdotai/ModernBERT-base (or run one load with allow_download=True).
2. Catch KompressModelNotCached and defer Kompress, falling back to a non-neural compressor.
3. Include the tokenizer files in the baked HF cache image alongside the model artifacts.

Example fix

# before
load_kompress_model(model_id, allow_download=False)  # tokenizer not cached -> raises

# after (deploy step)
# huggingface-cli download answerdotai/ModernBERT-base --local-dir $HF_HOME/hub/...
try:
    load_kompress_model(model_id, allow_download=False)
except KompressModelNotCached:
    use_fallback_compressor()

try:
    load_kompress_model(model_id, allow_download=False)
except KompressModelNotCached:
    use_fallback_compressor()  # tokenizer not cached; defer to a warm-up window

When it happens

Trigger: Loading Kompress in cache-only mode (allow_download=False) on a machine whose HF cache has the model weights but not the answerdotai/ModernBERT-base tokenizer files (or a completely cold cache).

Common situations: Offline deployments that cached the ONNX model via a different tool but never fetched the tokenizer; partial caches from interrupted downloads; containers built from a weights-only cache mount.

Related errors

• {model_id}
• offline mode (HEADROOM_BINARIES_OFFLINE=1) but fetch require
• Encoding not available for {self.model} - tokenizer {self.to
• Decoding not available for {self.model} - tokenizer {self.to
• No optimizer registered for '{key}'. Available: {available}

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/9036312f54bc1ff4. Report an issue: GitHub.