opendatalab/MinerU · error · InvalidConfig

default_prefix must be provided

Error message

default_prefix must be provided

What it means

MultiBucketS3Mixin.__init__ raises InvalidConfig when default_prefix is an empty string (`len(default_prefix) == 0`). The first path segment of default_prefix becomes self.default_bucket, so an empty prefix leaves the mixin without a default bucket and the constructor refuses to continue.

Source

Thrown at mineru/data/data_reader_writer/multi_bucket_s3.py:35

    return S3Reader, S3Writer


class MultiS3Mixin:
    def __init__(self, default_prefix: str, s3_configs: list[S3Config]):
        """Initialized with multiple s3 configs.

        Args:
            default_prefix (str): the default prefix of the relative path. for example, {some_bucket}/{some_prefix} or {some_bucket}
            s3_configs (list[S3Config]): list of s3 configs, the bucket_name must be unique in the list.

        Raises:
            InvalidConfig: default bucket config not in s3_configs.
            InvalidConfig: bucket name not unique in s3_configs.
            InvalidConfig: default bucket must be provided.
        """
        if len(default_prefix) == 0:
            raise InvalidConfig('default_prefix must be provided')

        arr = default_prefix.strip('/').split('/')
        self.default_bucket = arr[0]
        self.default_prefix = '/'.join(arr[1:])

        found_default_bucket_config = False
        for conf in s3_configs:
            if conf.bucket_name == self.default_bucket:
                found_default_bucket_config = True
                break

        if not found_default_bucket_config:
            raise InvalidConfig(
                f'default_bucket: {self.default_bucket} config must be provided in s3_configs: {s3_configs}'
            )

        uniq_bucket = set([conf.bucket_name for conf in s3_configs])
        if len(uniq_bucket) != len(s3_configs):

View on GitHub (pinned to 4fe4bde114)

Solutions

1. Pass a non-empty default_prefix of the form '{bucket}' or '{bucket}/{prefix}', e.g. 'my-bucket' or 'my-bucket/docs'.
2. Check the code that produces default_prefix (env var, CLI arg, config file) and supply a real bucket name.
3. Add a startup assertion so an empty prefix fails with your own message instead of the library's.

Example fix

# before
reader = MultiBucketS3DataReader('', s3_configs)

# after
reader = MultiBucketS3DataReader('my-bucket/pdfs', s3_configs)

def make_s3_reader(default_prefix: str, configs: list):
    if not default_prefix or not default_prefix.strip('/'):
        raise ValueError('default_prefix must be like "{bucket}" or "{bucket}/{prefix}"')
    return MultiBucketS3DataReader(default_prefix, configs)

When it happens

Trigger: Calling MultiBucketS3DataReader('' , configs) or MultiBucketS3DataWriter('', configs); also passing a whitespace-only string (len check does not strip first, though strip('/') happens after).

Common situations: default_prefix built dynamically from an env var or config key that is unset/empty; template strings like f'{bucket}/{prefix}' where both parts are empty; misconfigured pipeline YAML.

Related errors

• default_bucket: {self.default_bucket} config must be provide
• the bucket_name in s3_configs: {s3_configs} must be unique
• bucket name: {bucket_name} not found in s3_configs: {self.s3
• The provided path starts with '/'. This does not conform to
• Invalid S3 path format. Expected 's3://bucket-name/key' or '

AI-assisted analysis of opendatalab/MinerU@4fe4bde114 (2026-08-14). Data as JSON: /api/errors/c34593cfc6e864c8. Report an issue: GitHub.