headroomlabs-ai/headroom · error · KompressModelNotCached

{model_id}

Error message

{model_id}

What it means

This is KompressModelNotCached(model_id) (a RuntimeError subclass defined at kompress_compressor.py:138), raised when allow_download=False and every ONNX filename candidate was a cache miss (_NOT_CACHED_ERRORS = (OSError,)). It exists so cache-only callers (e.g. deferred model warm-up) can distinguish 'not downloaded yet' from a real load failure and avoid triggering network downloads.

Source

Thrown at headroom/transforms/kompress_compressor.py:685

            ort = onnxruntime
        try:
            session = ort.InferenceSession(
                onnx_path,
                _onnx_session_options(ort),
                providers=providers,
            )
            _smoke_run(session)
            return session
        except Exception as exc:
            last_err = exc
            logger.warning(
                "ONNX artifact %r from %s is unusable (%s); trying next candidate",
                filename,
                model_id,
                exc,
            )
    if not allow_download and cache_miss:
        raise KompressModelNotCached(model_id) from last_err
    raise FileNotFoundError(
        f"No loadable ONNX artifact in {model_id}; tried {_onnx_filename_candidates()}"
    ) from last_err


def _load_kompress_onnx(
    model_id: str,
    *,
    use_coreml: bool = False,
    allow_download: bool = True,
) -> tuple[Any, Any, str]:
    """Download ONNX INT8 model from HuggingFace and load with onnxruntime.

    When ``allow_download`` is ``False`` the model and tokenizer are loaded from
    the local cache only; a cache miss raises :class:`KompressModelNotCached`
    instead of hitting the network.
    """
    with _kompress_lock:

View on GitHub (pinned to 322425c43b)

Solutions

1. Pre-warm the cache once with downloads allowed (run a load with allow_download=True, or run huggingface-cli download <model_id> in an image-build step).
2. Catch KompressModelNotCached at the caller and defer/skip Kompress compression for this request instead of failing it.
3. Bake the HF model cache into the Docker image and point HF_HOME at it.

Example fix

# before
load_kompress_model(model_id, allow_download=False)  # raises KompressModelNotCached

# after
try:
    load_kompress_model(model_id, allow_download=False)
except KompressModelNotCached:
    use_fallback_compressor()  # defer; or warm cache at deploy time

try:
    model = load_kompress_model(model_id, allow_download=False)
except KompressModelNotCached:
    logger.info("Kompress model %s not cached; deferring", model_id)
    model = None  # defer warm-up; use fallback compressors

When it happens

Trigger: Loading the Kompress model with allow_download=False (cache-only mode) before the model artifacts were ever downloaded from HuggingFace in that environment.

Common situations: Offline or egress-restricted deployments; a fresh container/CI runner with a cold HF cache; calling a warm-up API that deliberately forbids network and expects to defer.

Related errors

• answerdotai/ModernBERT-base
• offline mode (HEADROOM_BINARIES_OFFLINE=1) but fetch require
• No loadable ONNX artifact in {model_id}; tried {_onnx_filena
• merged.pt for {model_id} is missing {missing_sections}; foun
• No optimizer registered for '{key}'. Available: {available}

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/e190848f9f021475. Report an issue: GitHub.