{"record":{"id":"581fdf9f3017d26b","repo":"opendatalab/MinerU","slug":"default-bucket-self-default-bucket-config-must","errorCode":null,"errorMessage":"default_bucket: {self.default_bucket} config must be provided in s3_configs: {s3_configs}","messagePattern":"default_bucket: (.+?) config must be provided in s3_configs: (.+?)","errorType":"validation","errorClass":"InvalidConfig","httpStatus":null,"severity":"error","filePath":"mineru/data/data_reader_writer/multi_bucket_s3.py","lineNumber":48,"sourceCode":"            InvalidConfig: default bucket config not in s3_configs.\n            InvalidConfig: bucket name not unique in s3_configs.\n            InvalidConfig: default bucket must be provided.\n        \"\"\"\n        if len(default_prefix) == 0:\n            raise InvalidConfig('default_prefix must be provided')\n\n        arr = default_prefix.strip('/').split('/')\n        self.default_bucket = arr[0]\n        self.default_prefix = '/'.join(arr[1:])\n\n        found_default_bucket_config = False\n        for conf in s3_configs:\n            if conf.bucket_name == self.default_bucket:\n                found_default_bucket_config = True\n                break\n\n        if not found_default_bucket_config:\n            raise InvalidConfig(\n                f'default_bucket: {self.default_bucket} config must be provided in s3_configs: {s3_configs}'\n            )\n\n        uniq_bucket = set([conf.bucket_name for conf in s3_configs])\n        if len(uniq_bucket) != len(s3_configs):\n            raise InvalidConfig(\n                f'the bucket_name in s3_configs: {s3_configs} must be unique'\n            )\n\n        self.s3_configs = s3_configs\n        self._s3_clients_h: dict = {}\n\n\nclass MultiBucketS3DataReader(DataReader, MultiS3Mixin):\n    def read(self, path: str) -> bytes:\n        \"\"\"Read the path from s3, select diffect bucket client for each request\n        based on the bucket, also support range read.\n","sourceCodeStart":30,"sourceCodeEnd":66,"githubUrl":"https://github.com/opendatalab/MinerU/blob/4fe4bde114a23ee5dd637eae99b767f4669bf58c/mineru/data/data_reader_writer/multi_bucket_s3.py#L30-L66","documentation":"InvalidConfig raised when the bucket named by the first segment of default_prefix has no matching S3Config in s3_configs. The mixin needs credentials for the default bucket, so it scans s3_configs for conf.bucket_name == self.default_bucket and aborts when none matches.","triggerScenarios":"MultiBucketS3DataReader('bucket-a/...', [S3Config(bucket_name='bucket-b', ...)]) — the default bucket 'bucket-a' is absent from the config list. Also triggered by leading/trailing whitespace or a '/' suffix on the bucket segment (only strip('/') is applied).","commonSituations":"Copy-paste mismatch between the default prefix and the credentials list; renaming a bucket in one place but not the other; per-environment configs (dev bucket list used with prod prefix); typos in bucket names.","solutions":["Add an S3Config whose bucket_name exactly equals the first segment of default_prefix.","Check for typos/case differences between default_prefix's bucket segment and the S3Config bucket_name values (matching is exact).","Print the parsed default bucket (`'my-prefix'.strip('/').split('/')[0]`) and diff it against `[c.bucket_name for c in s3_configs]` to find the mismatch."],"exampleFix":"# before\nreader = MultiBucketS3DataReader(\n    'docs-bucket/pdfs',\n    [S3Config(bucket_name='archive-bucket', ...)],\n)\n\n# after\nreader = MultiBucketS3DataReader(\n    'docs-bucket/pdfs',\n    [\n        S3Config(bucket_name='docs-bucket', ...),\n        S3Config(bucket_name='archive-bucket', ...),\n    ],\n)","handlingStrategy":"validation","validationCode":"from mineru.data.utils.exceptions import InvalidConfig\n\ndef validate_s3_setup(default_prefix: str, configs: list) -> None:\n    default_bucket = default_prefix.strip('/').split('/')[0]\n    names = [c.bucket_name for c in configs]\n    if not default_bucket:\n        raise ValueError('default_prefix must contain a bucket segment')\n    if default_bucket not in names:\n        raise ValueError(f'add S3Config for default bucket {default_bucket!r}; have {names}')","typeGuard":null,"tryCatchPattern":"try:\n    reader = MultiBucketS3DataReader(prefix, configs)\nexcept InvalidConfig as e:\n    raise RuntimeError(f'S3 setup invalid: {e.msg}') from e","preventionTips":["Generate the default bucket's S3Config from the same variable used to build default_prefix.","Lift bucket names from one constants module instead of duplicating strings in configs and prefixes.","Log the parsed default bucket next to the configured bucket list at startup."],"tags":["s3","config","validation","constructor"],"backgroundTag":null,"analyzedSha":"4fe4bde114a23ee5dd637eae99b767f4669bf58c","analyzedAt":"2026-08-14T21:29:18.456Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}