opendatalab/MinerU · error · ModuleNotFoundError

S3 reader/writer requires optional dependencies. Install the

Error message

S3 reader/writer requires optional dependencies. Install them with `pip install 'mineru[s3]'`.

What it means

Raised by MultiBucketS3Mixin's lazy loader when `mineru.io.s3` cannot be imported because boto3/botocore are not installed. MinerU ships S3 support as an optional extra, so the boto3 stack is only required at the moment an S3 reader/writer actually tries to read or write. The original ModuleNotFoundError is chained (`from exc`) so the root missing module is visible in the traceback.

Source

Thrown at mineru/data/data_reader_writer/multi_bucket_s3.py:14

# Copyright (c) Opendatalab. All rights reserved.

from ..utils.exceptions import InvalidConfig, InvalidParams
from .base import DataReader, DataWriter
from ..utils.schemas import S3Config
from ..utils.path_utils import parse_s3_range_params, parse_s3path, remove_non_official_s3_args


def _load_s3_io_classes():
    """延迟加载 S3 IO 类，仅在实际读写 S3 时要求安装 boto3。"""
    try:
        from ..io.s3 import S3Reader, S3Writer
    except ModuleNotFoundError as exc:
        raise ModuleNotFoundError(
            "S3 reader/writer requires optional dependencies. Install them with `pip install 'mineru[s3]'`."
        ) from exc

    return S3Reader, S3Writer


class MultiS3Mixin:
    def __init__(self, default_prefix: str, s3_configs: list[S3Config]):
        """Initialized with multiple s3 configs.

        Args:
            default_prefix (str): the default prefix of the relative path. for example, {some_bucket}/{some_prefix} or {some_bucket}
            s3_configs (list[S3Config]): list of s3 configs, the bucket_name must be unique in the list.

        Raises:
            InvalidConfig: default bucket config not in s3_configs.
            InvalidConfig: bucket name not unique in s3_configs.
            InvalidConfig: default bucket must be provided.

View on GitHub (pinned to 4fe4bde114)

Solutions

Install the S3 extra: `pip install 'mineru[s3]'` (or add it to requirements/pyproject).
Alternatively install the underlying deps directly: `pip install boto3 botocore`.
Verify with `python -c "import boto3; from botocore.config import Config"` before rerunning the S3 job.
If you never use S3, switch the reader/writer back to local paths so the lazy import is never triggered.

Example fix

# before (base install only)
pip install mineru
reader = MultiBucketS3DataReader('bucket/prefix', [cfg])
reader.read('s3://bucket/doc.pdf')  # ModuleNotFoundError

# after
pip install 'mineru[s3]'
reader.read('s3://bucket/doc.pdf')

Defensive patterns

Strategy: validation

Validate before calling

def s3_deps_available() -> bool:
    try:
        import boto3  # noqa: F401
        from botocore.config import Config  # noqa: F401
        return True
    except ImportError:
        return False

if not s3_deps_available():
    raise SystemExit("Install S3 support first: pip install 'mineru[s3]'")

Try / catch

try:
    reader = MultiBucketS3DataReader('bkt/prefix', configs)
    data = reader.read('s3://bkt/doc.pdf')
except ModuleNotFoundError as e:
    if "mineru[s3]" in str(e):
        raise SystemExit("Missing S3 extra. Run: pip install 'mineru[s3]'") from e
    raise

Prevention

Declare 'mineru[s3]' in requirements/pyproject for any deployment that touches s3:// paths.
Run a boto3 import check during deployment smoke tests before processing data.
Treat any s3:// path in configuration as a signal that the [s3] extra is required.

When it happens

Trigger: Constructing MultiBucketS3DataReader/MultiBucketS3DataWriter with S3Config entries and then calling read()/read_at()/write() — the first __get_s3_client() call invokes _load_s3_io_classes(), whose `from ..io.s3 import S3Reader, S3Writer` fails when boto3 is absent.

Common situations: Default `pip install mineru` install (no [s3] extra) combined with s3:// input paths in CLI or SDK usage; CI images built from the base package; deployments that only later switch output_writer to an S3 writer.

Related errors

AI-assisted analysis of opendatalab/MinerU@4fe4bde114 (2026-08-14). Data as JSON: /api/errors/c6eb8d6e383c3af9. Report an issue: GitHub.