docling-project/docling · error · ImportError

rednote-hilab/dots.mocr requires flash-attn with the Transfo

Error message

rednote-hilab/dots.mocr requires flash-attn with the Transformers engine. Install flash-attn in the transformers-v4 environment before using this model.

What it means

The rednote-hilab/dots.mocr model requires flash-attention kernels with the Transformers engine; Docling probes for the flash_attn module at model construction and raises ImportError (chained from the real ImportError) when it is absent. This check only fires for dots.mocr — plain dots.ocr does not require it.

Source

Thrown at docling/models/vlm_pipeline_models/hf_transformers_model.py:44

from docling.models.utils.generation_utils import GenerationStopper
from docling.models.utils.hf_model_download import (
    HuggingFaceModelDownloadMixin,
)
from docling.models.utils.hf_stopping_criteria import HFStoppingCriteriaWrapper
from docling.utils.accelerator_utils import decide_device
from docling.utils.profiling import TimeRecorder

_log = logging.getLogger(__name__)

_DOTS_REPO_IDS = {"rednote-hilab/dots.ocr", "rednote-hilab/dots.mocr"}
_DOTS_FLASH_ATTN_REQUIRED_REPO_IDS = {"rednote-hilab/dots.mocr"}


def _ensure_dots_flash_attn_import() -> None:
    try:
        importlib.import_module("flash_attn")
    except ImportError as exc:
        raise ImportError(
            "rednote-hilab/dots.mocr requires flash-attn with the Transformers "
            "engine. Install flash-attn in the transformers-v4 environment "
            "before using this model."
        ) from exc


class HuggingFaceTransformersVlmModel(BaseVlmPageModel, HuggingFaceModelDownloadMixin):
    def __init__(
        self,
        enabled: bool,
        artifacts_path: Path | None,
        accelerator_options: AcceleratorOptions,
        vlm_options: InlineVlmOptions,
    ):
        self.enabled = enabled

        self.vlm_options = vlm_options

View on GitHub (pinned to 61d76f1ff3)

Solutions

Install a flash-attn wheel matching your torch/CUDA version in the transformers-v4 environment: pip install flash-attn --no-build-isolation (or use a prebuilt wheel)
If you cannot install flash-attn, switch repo_id to rednote-hilab/dots.ocr which has no such requirement
Alternatively drive dots.mocr through the vLLM engine, which manages its own attention backend

Example fix

# before
vlm_options.repo_id = "rednote-hilab/dots.mocr"  # ImportError: needs flash_attn
# after (option 1)
# pip install flash-attn --no-build-isolation
# after (option 2)
vlm_options.repo_id = "rednote-hilab/dots.ocr"

Defensive patterns

Strategy: validation

Validate before calling

import importlib.util

if vlm_options.repo_id == 'rednote-hilab/dots.mocr':
    if importlib.util.find_spec('flash_attn') is None:
        raise ImportError('dots.mocr needs flash-attn; install it or use dots.ocr')

Try / catch

try:
    model = HuggingFaceTransformersVlmModel(...)
except ImportError as e:
    if 'flash-attn' in str(e):
        vlm_options.repo_id = 'rednote-hilab/dots.ocr'
        model = HuggingFaceTransformersVlmModel(...)
    else:
        raise

Prevention

Pin torch and install a matching flash-attn wheel when provisioning the transformers-v4 env
Probe importlib.util.find_spec('flash_attn') at app startup before model selection
Document per-model extras (dots.mocr -> flash-attn) next to your model registry

When it happens

Trigger: Setting VlmOptions repo_id='rednote-hilab/dots.mocr' with the Transformers engine in an environment where flash_attn is not installed (it is not a default docling dependency and often needs matching CUDA/torch versions).

Common situations: Using docling in the recommended transformers-v4 environment but flash-attn was never built there; upgrading torch so the previously compiled flash-attn wheel no longer imports; CPU-only environments where flash-attn is impractical.

Related errors

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/0e132ddfaafe74f5. Report an issue: GitHub.