{"record":{"id":"c6eb8d6e383c3af9","repo":"opendatalab/MinerU","slug":"s3-reader-writer-requires-optional-dependencies-i","errorCode":null,"errorMessage":"S3 reader/writer requires optional dependencies. Install them with `pip install 'mineru[s3]'`.","messagePattern":"S3 reader/writer requires optional dependencies\\. Install them with `pip install 'mineru\\[s3\\]'`\\.","errorType":"exception","errorClass":"ModuleNotFoundError","httpStatus":null,"severity":"error","filePath":"mineru/data/data_reader_writer/multi_bucket_s3.py","lineNumber":14,"sourceCode":"# Copyright (c) Opendatalab. All rights reserved.\n\nfrom ..utils.exceptions import InvalidConfig, InvalidParams\nfrom .base import DataReader, DataWriter\nfrom ..utils.schemas import S3Config\nfrom ..utils.path_utils import parse_s3_range_params, parse_s3path, remove_non_official_s3_args\n\n\ndef _load_s3_io_classes():\n    \"\"\"延迟加载 S3 IO 类，仅在实际读写 S3 时要求安装 boto3。\"\"\"\n    try:\n        from ..io.s3 import S3Reader, S3Writer\n    except ModuleNotFoundError as exc:\n        raise ModuleNotFoundError(\n            \"S3 reader/writer requires optional dependencies. Install them with `pip install 'mineru[s3]'`.\"\n        ) from exc\n\n    return S3Reader, S3Writer\n\n\nclass MultiS3Mixin:\n    def __init__(self, default_prefix: str, s3_configs: list[S3Config]):\n        \"\"\"Initialized with multiple s3 configs.\n\n        Args:\n            default_prefix (str): the default prefix of the relative path. for example, {some_bucket}/{some_prefix} or {some_bucket}\n            s3_configs (list[S3Config]): list of s3 configs, the bucket_name must be unique in the list.\n\n        Raises:\n            InvalidConfig: default bucket config not in s3_configs.\n            InvalidConfig: bucket name not unique in s3_configs.\n            InvalidConfig: default bucket must be provided.","sourceCodeStart":1,"sourceCodeEnd":32,"githubUrl":"https://github.com/opendatalab/MinerU/blob/4fe4bde114a23ee5dd637eae99b767f4669bf58c/mineru/data/data_reader_writer/multi_bucket_s3.py#L1-L32","documentation":"Raised by MultiBucketS3Mixin's lazy loader when `mineru.io.s3` cannot be imported because boto3/botocore are not installed. MinerU ships S3 support as an optional extra, so the boto3 stack is only required at the moment an S3 reader/writer actually tries to read or write. The original ModuleNotFoundError is chained (`from exc`) so the root missing module is visible in the traceback.","triggerScenarios":"Constructing MultiBucketS3DataReader/MultiBucketS3DataWriter with S3Config entries and then calling read()/read_at()/write() — the first __get_s3_client() call invokes _load_s3_io_classes(), whose `from ..io.s3 import S3Reader, S3Writer` fails when boto3 is absent.","commonSituations":"Default `pip install mineru` install (no [s3] extra) combined with s3:// input paths in CLI or SDK usage; CI images built from the base package; deployments that only later switch output_writer to an S3 writer.","solutions":["Install the S3 extra: `pip install 'mineru[s3]'` (or add it to requirements/pyproject).","Alternatively install the underlying deps directly: `pip install boto3 botocore`.","Verify with `python -c \"import boto3; from botocore.config import Config\"` before rerunning the S3 job.","If you never use S3, switch the reader/writer back to local paths so the lazy import is never triggered."],"exampleFix":"# before (base install only)\npip install mineru\nreader = MultiBucketS3DataReader('bucket/prefix', [cfg])\nreader.read('s3://bucket/doc.pdf')  # ModuleNotFoundError\n\n# after\npip install 'mineru[s3]'\nreader.read('s3://bucket/doc.pdf')","handlingStrategy":"validation","validationCode":"def s3_deps_available() -> bool:\n    try:\n        import boto3  # noqa: F401\n        from botocore.config import Config  # noqa: F401\n        return True\n    except ImportError:\n        return False\n\nif not s3_deps_available():\n    raise SystemExit(\"Install S3 support first: pip install 'mineru[s3]'\")","typeGuard":null,"tryCatchPattern":"try:\n    reader = MultiBucketS3DataReader('bkt/prefix', configs)\n    data = reader.read('s3://bkt/doc.pdf')\nexcept ModuleNotFoundError as e:\n    if \"mineru[s3]\" in str(e):\n        raise SystemExit(\"Missing S3 extra. Run: pip install 'mineru[s3]'\") from e\n    raise","preventionTips":["Declare 'mineru[s3]' in requirements/pyproject for any deployment that touches s3:// paths.","Run a boto3 import check during deployment smoke tests before processing data.","Treat any s3:// path in configuration as a signal that the [s3] extra is required."],"tags":["s3","dependencies","optional-deps","boto3","lazy-import"],"backgroundTag":null,"analyzedSha":"4fe4bde114a23ee5dd637eae99b767f4669bf58c","analyzedAt":"2026-08-14T21:29:18.456Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}