docling-project/docling · error · ImportError

rednote-hilab/dots.mocr requires flash-attn with the Transfo

Error message

rednote-hilab/dots.mocr requires flash-attn with the Transformers engine. Install flash-attn in the transformers-v4 environment before using this model.

What it means

Certain Dots OCR models (repo ids in _DOTS_FLASH_ATTN_REQUIRED_REPO_IDS, e.g. rednote-hilab/dots.mocr) need flash-attn when run through the Transformers engine. During model validation the engine imports flash_attn and converts its absence into an ImportError with install guidance, because generation quality/stability is not guaranteed without it.

Source

Thrown at docling/models/inference_engines/vlm/transformers_engine.py:66

    from docling.datamodel.stage_model_specs import EngineModelConfig

_log = logging.getLogger(__name__)

_DOTS_REPO_IDS = {"rednote-hilab/dots.ocr", "rednote-hilab/dots.mocr"}
_DOTS_FLASH_ATTN_REQUIRED_REPO_IDS = {"rednote-hilab/dots.mocr"}


def _coerce_transformers_model_type(value: Any) -> TransformersModelType:
    if isinstance(value, TransformersModelType):
        return value
    return TransformersModelType(value)


def _ensure_dots_flash_attn_import() -> None:
    try:
        importlib.import_module("flash_attn")
    except ImportError as exc:
        raise ImportError(
            "rednote-hilab/dots.mocr requires flash-attn with the Transformers "
            "engine. Install flash-attn in the transformers-v4 environment "
            "before using this model."
        ) from exc


class TransformersVlmEngine(BaseVlmEngine, HuggingFaceModelDownloadMixin):
    """HuggingFace Transformers engine for VLM inference.

    This engine uses the transformers library to run vision-language models
    locally on CPU, CUDA, or XPU devices.
    """

    def __init__(
        self,
        options: TransformersVlmEngineOptions,
        accelerator_options: AcceleratorOptions,
        artifacts_path: Union[Path, str] | None,

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Install flash-attn in the transformers-v4 environment (pip install flash-attn --no-build-isolation, or use a prebuilt wheel matching your torch/CUDA version)
2. Verify the import works: python -c 'import flash_attn; print(flash_attn.__version__)'
3. If flash-attn cannot be installed, run the same Dots model through the vLLM engine instead (it supplies its own attention backend)

Example fix

# before (env without flash-attn)
options = TransformersVlmEngineOptions()
model_config = EngineModelConfig(repo_id='rednote-hilab/dots.mocr')
# engine = TransformersVlmEngine(...)  -> ImportError at initialize

# after
# terminal: pip install flash-attn --no-build-isolation
engine = TransformersVlmEngine(options=options, model_config=model_config, ...)

def flash_attn_available() -> bool:
    try:
        import flash_attn  # noqa: F401
        return True
    except ImportError:
        return False

DOTS_FLASH_ATTN_MODELS = {'rednote-hilab/dots.mocr'}
if repo_id in DOTS_FLASH_ATTN_MODELS and not flash_attn_available():
    raise SystemExit(f'{repo_id} needs flash-attn on the Transformers engine; pip install flash-attn --no-build-isolation')

try:
    engine.initialize()
except ImportError as e:
    if 'flash-attn' in str(e):
        raise SystemExit('Install flash-attn, or run this Dots model on the vLLM engine instead') from e
    raise

When it happens

Trigger: Configuring TransformersVlmEngine (or a VLM pipeline) with repo_id in _DOTS_FLASH_ATTN_REQUIRED_REPO_IDS in an environment where the flash_attn module cannot be imported; the check runs during initialize()/model validation, before any weights load.

Common situations: Using the default dots.mocr model spec on a fresh venv without flash-attn; flash-attn failing to install (it needs a matching CUDA toolchain and often long compile times); running on CPU-only boxes where flash-attn wheels are unavailable.

Related errors

• {repo_id} is supported by the Transformers engine only with
• mlx-vlm is not installed. Please install it via `pip install
• Phi 4 only works with transformers<4.52.0 but you have {tran
• vLLM is not installed. Please install it via `pip install vl
• transformers >=4.46 is not installed. Please install Docling

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/50b0c3abb7e4a737. Report an issue: GitHub.