opendatalab/MinerU · error · InvalidConfig

default_bucket: {self.default_bucket} config must be provide

Error message

default_bucket: {self.default_bucket} config must be provided in s3_configs: {s3_configs}

What it means

InvalidConfig raised when the bucket named by the first segment of default_prefix has no matching S3Config in s3_configs. The mixin needs credentials for the default bucket, so it scans s3_configs for conf.bucket_name == self.default_bucket and aborts when none matches.

Source

Thrown at mineru/data/data_reader_writer/multi_bucket_s3.py:48

            InvalidConfig: default bucket config not in s3_configs.
            InvalidConfig: bucket name not unique in s3_configs.
            InvalidConfig: default bucket must be provided.
        """
        if len(default_prefix) == 0:
            raise InvalidConfig('default_prefix must be provided')

        arr = default_prefix.strip('/').split('/')
        self.default_bucket = arr[0]
        self.default_prefix = '/'.join(arr[1:])

        found_default_bucket_config = False
        for conf in s3_configs:
            if conf.bucket_name == self.default_bucket:
                found_default_bucket_config = True
                break

        if not found_default_bucket_config:
            raise InvalidConfig(
                f'default_bucket: {self.default_bucket} config must be provided in s3_configs: {s3_configs}'
            )

        uniq_bucket = set([conf.bucket_name for conf in s3_configs])
        if len(uniq_bucket) != len(s3_configs):
            raise InvalidConfig(
                f'the bucket_name in s3_configs: {s3_configs} must be unique'
            )

        self.s3_configs = s3_configs
        self._s3_clients_h: dict = {}


class MultiBucketS3DataReader(DataReader, MultiS3Mixin):
    def read(self, path: str) -> bytes:
        """Read the path from s3, select diffect bucket client for each request
        based on the bucket, also support range read.

View on GitHub (pinned to 4fe4bde114)

Solutions

1. Add an S3Config whose bucket_name exactly equals the first segment of default_prefix.
2. Check for typos/case differences between default_prefix's bucket segment and the S3Config bucket_name values (matching is exact).
3. Print the parsed default bucket (`'my-prefix'.strip('/').split('/')[0]`) and diff it against `[c.bucket_name for c in s3_configs]` to find the mismatch.

Example fix

# before
reader = MultiBucketS3DataReader(
    'docs-bucket/pdfs',
    [S3Config(bucket_name='archive-bucket', ...)],
)

# after
reader = MultiBucketS3DataReader(
    'docs-bucket/pdfs',
    [
        S3Config(bucket_name='docs-bucket', ...),
        S3Config(bucket_name='archive-bucket', ...),
    ],
)

from mineru.data.utils.exceptions import InvalidConfig

def validate_s3_setup(default_prefix: str, configs: list) -> None:
    default_bucket = default_prefix.strip('/').split('/')[0]
    names = [c.bucket_name for c in configs]
    if not default_bucket:
        raise ValueError('default_prefix must contain a bucket segment')
    if default_bucket not in names:
        raise ValueError(f'add S3Config for default bucket {default_bucket!r}; have {names}')

try:
    reader = MultiBucketS3DataReader(prefix, configs)
except InvalidConfig as e:
    raise RuntimeError(f'S3 setup invalid: {e.msg}') from e

When it happens

Trigger: MultiBucketS3DataReader('bucket-a/...', [S3Config(bucket_name='bucket-b', ...)]) — the default bucket 'bucket-a' is absent from the config list. Also triggered by leading/trailing whitespace or a '/' suffix on the bucket segment (only strip('/') is applied).

Common situations: Copy-paste mismatch between the default prefix and the credentials list; renaming a bucket in one place but not the other; per-environment configs (dev bucket list used with prod prefix); typos in bucket names.

Related errors

• default_prefix must be provided
• the bucket_name in s3_configs: {s3_configs} must be unique
• bucket name: {bucket_name} not found in s3_configs: {self.s3
• The provided path starts with '/'. This does not conform to
• Invalid S3 path format. Expected 's3://bucket-name/key' or '

AI-assisted analysis of opendatalab/MinerU@4fe4bde114 (2026-08-14). Data as JSON: /api/errors/581fdf9f3017d26b. Report an issue: GitHub.