{"record":{"id":"3cb269b86c71957d","repo":"infiniflow/ragflow","slug":"s3-compatible-storage","errorCode":null,"errorMessage":"S3 Compatible Storage","messagePattern":"S3 Compatible Storage","errorType":"validation","errorClass":"ConnectorMissingCredentialError","httpStatus":null,"severity":"error","filePath":"common/data_source/blob_connector.py","lineNumber":115,"sourceCode":"                    raise ConnectorMissingCredentialError(\"Amazon S3 IAM role ARN is required\")\n\n            elif authentication_method == \"assume_role\":\n                pass\n\n            else:\n                raise ConnectorMissingCredentialError(\"Unsupported S3 authentication method\")\n\n        elif self.bucket_type == BlobType.GOOGLE_CLOUD_STORAGE:\n            if not all(credentials.get(key) for key in [\"access_key_id\", \"secret_access_key\"]):\n                raise ConnectorMissingCredentialError(\"Google Cloud Storage\")\n\n        elif self.bucket_type == BlobType.OCI_STORAGE:\n            if not all(credentials.get(key) for key in [\"namespace\", \"region\", \"access_key_id\", \"secret_access_key\"]):\n                raise ConnectorMissingCredentialError(\"Oracle Cloud Infrastructure\")\n\n        elif self.bucket_type == BlobType.S3_COMPATIBLE:\n            if not all(credentials.get(key) for key in [\"endpoint_url\", \"aws_access_key_id\", \"aws_secret_access_key\", \"addressing_style\"]):\n                raise ConnectorMissingCredentialError(\"S3 Compatible Storage\")\n\n        else:\n            raise ValueError(f\"Unsupported bucket type: {self.bucket_type}\")\n\n        # Create S3 client\n        self.s3_client = create_s3_client(self.bucket_type, credentials, self.european_residency)\n\n        # Detect bucket region (only important for S3)\n        if self.bucket_type == BlobType.S3:\n            self.bucket_region = detect_bucket_region(self.s3_client, self.bucket_name)\n\n        return None\n\n    def _build_document_from_obj(\n        self,\n        obj: dict[str, Any],\n        filename_counts: dict[str, int],\n    ) -> Optional[Document]:","sourceCodeStart":97,"sourceCodeEnd":133,"githubUrl":"https://github.com/infiniflow/ragflow/blob/554fb1133ac3861732235ad9c377eb5e0a770665/common/data_source/blob_connector.py#L97-L133","documentation":"Raised by BlobStorageConnector.load_credentials when bucket_type is 's3_compatible' and the credentials dict is missing any of: endpoint_url, aws_access_key_id, aws_secret_access_key, addressing_style. S3-compatible providers (MinIO, Ceph, Wasabi, DigitalOcean Spaces, etc.) need an explicit endpoint and addressing style ('path' or 'virtual') in addition to the HMAC key pair, because boto3 cannot guess the endpoint from the key alone.","triggerScenarios":"Calling load_credentials with bucket_type='s3_compatible' and a credentials dict that has AWS-style keys but no endpoint_url or addressing_style. Also triggered when endpoint_url is '' or None, or when addressing_style is omitted so the client cannot decide between path-style and virtual-host-style URLs.","commonSituations":"Pointing the connector at a self-hosted MinIO or a third-party S3 clone and reusing the plain S3 credential shape; setting endpoint_url without the scheme; forgetting addressing_style ('path' is required by MinIO and most on-prem deployments, while 'virtual' is needed by some providers).","solutions":["Add endpoint_url (e.g. 'https://minio.example.com:9000') and addressing_style ('path' or 'virtual') to the credentials dict","Confirm aws_access_key_id and aws_secret_access_key are the provider's S3-compatible keys, not IAM role config","Use addressing_style='path' for MinIO/Ceph/local deployments; 'virtual' only if the provider requires virtual-hosted buckets","Include the scheme (http/https) in endpoint_url since it is passed straight into the boto3 endpoint_url parameter"],"exampleFix":"// before\nconnector.load_credentials({\n    'aws_access_key_id': key,\n    'aws_secret_access_key': secret,\n})\n// after\nconnector.load_credentials({\n    'endpoint_url': 'https://minio.example.com:9000',\n    'aws_access_key_id': key,\n    'aws_secret_access_key': secret,\n    'addressing_style': 'path',\n})","handlingStrategy":"validation","validationCode":"REQUIRED_S3C = ['endpoint_url', 'aws_access_key_id', 'aws_secret_access_key', 'addressing_style']\nif not all(credentials.get(k) for k in REQUIRED_S3C):\n    raise ValueError('s3_compatible needs endpoint_url, key pair, and addressing_style')\nif credentials['addressing_style'] not in ('path', 'virtual'):\n    raise ValueError(\"addressing_style must be 'path' or 'virtual'\")\nconnector.load_credentials(credentials)","typeGuard":"def has_s3_compatible_credentials(c: dict) -> bool:\n    return (isinstance(c.get('endpoint_url'), str)\n            and c['endpoint_url'].startswith(('http://', 'https://'))\n            and bool(c.get('aws_access_key_id'))\n            and bool(c.get('aws_secret_access_key'))\n            and c.get('addressing_style') in ('path', 'virtual'))","tryCatchPattern":"try:\n    connector.load_credentials(creds)\nexcept ConnectorMissingCredentialError as e:\n    if 'S3 Compatible' in str(e):\n        raise ConfigError('endpoint_url + aws key pair + addressing_style required') from e\n    raise","preventionTips":["Default addressing_style to 'path' in config templates — correct for MinIO and most self-hosted stores","Validate endpoint_url has a scheme at form-fill time; boto3 accepts scheme-less URLs and fails much later"],"tags":["s3","minio","credentials","blob-storage","configuration"],"backgroundTag":null,"analyzedSha":"554fb1133ac3861732235ad9c377eb5e0a770665","analyzedAt":"2026-08-15T09:20:16.380Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}