{"record":{"id":"50b0c3abb7e4a737","repo":"docling-project/docling","slug":"rednote-hilab-dots-mocr-requires-flash-attn-with-t","errorCode":null,"errorMessage":"rednote-hilab/dots.mocr requires flash-attn with the Transformers engine. Install flash-attn in the transformers-v4 environment before using this model.","messagePattern":"rednote-hilab/dots\\.mocr requires flash-attn with the Transformers engine\\. Install flash-attn in the transformers-v4 environment before using this model\\.","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"docling/models/inference_engines/vlm/transformers_engine.py","lineNumber":66,"sourceCode":"    from docling.datamodel.stage_model_specs import EngineModelConfig\n\n_log = logging.getLogger(__name__)\n\n_DOTS_REPO_IDS = {\"rednote-hilab/dots.ocr\", \"rednote-hilab/dots.mocr\"}\n_DOTS_FLASH_ATTN_REQUIRED_REPO_IDS = {\"rednote-hilab/dots.mocr\"}\n\n\ndef _coerce_transformers_model_type(value: Any) -> TransformersModelType:\n    if isinstance(value, TransformersModelType):\n        return value\n    return TransformersModelType(value)\n\n\ndef _ensure_dots_flash_attn_import() -> None:\n    try:\n        importlib.import_module(\"flash_attn\")\n    except ImportError as exc:\n        raise ImportError(\n            \"rednote-hilab/dots.mocr requires flash-attn with the Transformers \"\n            \"engine. Install flash-attn in the transformers-v4 environment \"\n            \"before using this model.\"\n        ) from exc\n\n\nclass TransformersVlmEngine(BaseVlmEngine, HuggingFaceModelDownloadMixin):\n    \"\"\"HuggingFace Transformers engine for VLM inference.\n\n    This engine uses the transformers library to run vision-language models\n    locally on CPU, CUDA, or XPU devices.\n    \"\"\"\n\n    def __init__(\n        self,\n        options: TransformersVlmEngineOptions,\n        accelerator_options: AcceleratorOptions,\n        artifacts_path: Union[Path, str] | None,","sourceCodeStart":48,"sourceCodeEnd":84,"githubUrl":"https://github.com/docling-project/docling/blob/61d76f1ff3f8428065465889f7b4577da7df704c/docling/models/inference_engines/vlm/transformers_engine.py#L48-L84","documentation":"Certain Dots OCR models (repo ids in _DOTS_FLASH_ATTN_REQUIRED_REPO_IDS, e.g. rednote-hilab/dots.mocr) need flash-attn when run through the Transformers engine. During model validation the engine imports flash_attn and converts its absence into an ImportError with install guidance, because generation quality/stability is not guaranteed without it.","triggerScenarios":"Configuring TransformersVlmEngine (or a VLM pipeline) with repo_id in _DOTS_FLASH_ATTN_REQUIRED_REPO_IDS in an environment where the flash_attn module cannot be imported; the check runs during initialize()/model validation, before any weights load.","commonSituations":"Using the default dots.mocr model spec on a fresh venv without flash-attn; flash-attn failing to install (it needs a matching CUDA toolchain and often long compile times); running on CPU-only boxes where flash-attn wheels are unavailable.","solutions":["Install flash-attn in the transformers-v4 environment (pip install flash-attn --no-build-isolation, or use a prebuilt wheel matching your torch/CUDA version)","Verify the import works: python -c 'import flash_attn; print(flash_attn.__version__)'","If flash-attn cannot be installed, run the same Dots model through the vLLM engine instead (it supplies its own attention backend)"],"exampleFix":"# before (env without flash-attn)\noptions = TransformersVlmEngineOptions()\nmodel_config = EngineModelConfig(repo_id='rednote-hilab/dots.mocr')\n# engine = TransformersVlmEngine(...)  -> ImportError at initialize\n\n# after\n# terminal: pip install flash-attn --no-build-isolation\nengine = TransformersVlmEngine(options=options, model_config=model_config, ...)","handlingStrategy":"validation","validationCode":"def flash_attn_available() -> bool:\n    try:\n        import flash_attn  # noqa: F401\n        return True\n    except ImportError:\n        return False\n\nDOTS_FLASH_ATTN_MODELS = {'rednote-hilab/dots.mocr'}\nif repo_id in DOTS_FLASH_ATTN_MODELS and not flash_attn_available():\n    raise SystemExit(f'{repo_id} needs flash-attn on the Transformers engine; pip install flash-attn --no-build-isolation')","typeGuard":null,"tryCatchPattern":"try:\n    engine.initialize()\nexcept ImportError as e:\n    if 'flash-attn' in str(e):\n        raise SystemExit('Install flash-attn, or run this Dots model on the vLLM engine instead') from e\n    raise","preventionTips":["Bake flash-attn (prebuilt wheel matching torch/CUDA) into images that run Dots OCR models","Verify 'import flash_attn' in environment health checks for Transformers-based Dots pipelines","Keep a vLLM-engine fallback path for hosts where flash-attn cannot compile"],"tags":["vlm","transformers","flash-attn","dependency","import-error","dots-ocr"],"backgroundTag":null,"analyzedSha":"61d76f1ff3f8428065465889f7b4577da7df704c","analyzedAt":"2026-08-14T23:53:18.727Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}