{"record":{"id":"24283e1146c3ffca","repo":"infiniflow/ragflow","slug":"blob-storage","errorCode":null,"errorMessage":"Blob storage","messagePattern":"Blob storage","errorType":"validation","errorClass":"ConnectorMissingCredentialError","httpStatus":null,"severity":"error","filePath":"common/data_source/blob_connector.py","lineNumber":192,"sourceCode":"                if len(batch) == self.batch_size:\n                    yield batch\n                    batch = []\n            except Exception:\n                logging.exception(f\"Error decoding object {obj.get('Key')}\")\n\n        if batch:\n            yield batch\n\n    def list_keys(self) -> Iterator[KeyRecord]:\n        \"\"\"Enumerate the full bucket keyspace with per-object fingerprints.\n\n        Cheap path: relies on list_objects_v2 which returns ETag in the listing,\n        so no GetObject call is needed. Caches each object's metadata so a\n        subsequent get_value(key) call can rebuild the Document without a second\n        round-trip to S3.\n        \"\"\"\n        if self.s3_client is None:\n            raise ConnectorMissingCredentialError(\"Blob storage\")\n\n        all_objects, filename_counts = self._collect_blob_objects(\n            start=datetime(1970, 1, 1, tzinfo=timezone.utc),\n            end=datetime.now(timezone.utc),\n        )\n        self._filename_counts = filename_counts\n        self._listing_cache = {}\n\n        for obj in all_objects:\n            doc_id = f\"{self.bucket_type}:{self.bucket_name}:{obj['Key']}\"\n            self._listing_cache[doc_id] = obj\n            yield KeyRecord(\n                key=doc_id,\n                fingerprint=_normalize_etag(obj.get(\"ETag\")),\n            )\n\n    def get_value(self, key: str) -> Document:\n        \"\"\"Materialize the Document for a key previously yielded by list_keys().","sourceCodeStart":174,"sourceCodeEnd":210,"githubUrl":"https://github.com/infiniflow/ragflow/blob/554fb1133ac3861732235ad9c377eb5e0a770665/common/data_source/blob_connector.py#L174-L210","documentation":"Raised by BlobStorageConnector.list_keys when self.s3_client is None, i.e. load_credentials was never called (or was called but failed before create_s3_client ran). list_keys enumerates the whole bucket keyspace with ETag fingerprints and cannot run without the boto3 client, so the connector treats a missing client as missing credentials. Message is the generic 'Blob storage'.","triggerScenarios":"Instantiating BlobStorageConnector and calling list_keys() directly without a prior successful load_credentials(credentials) call; or load_credentials raising ConnectorMissingCredentialError partway so s3_client was never assigned (it is only set at the end of the method).","commonSituations":"Indexing pipeline orchestrator skips or mishandles the credential-loading step; a failed load_credentials exception is swallowed upstream and the run continues to the fingerprint/listing phase; unit tests constructing the connector without credentials.","solutions":["Call connector.load_credentials(credentials) and let it complete successfully before list_keys()","Ensure the credential dict passes the per-bucket_type validation (R2/S3/GCS/OCI/S3-compatible key sets) so load_credentials reaches create_s3_client","Do not catch-and-continue on load_credentials errors in the orchestration code — treat them as fatal for the run","In tests, call load_credentials with dummy valid-shaped credentials before exercising list_keys"],"exampleFix":"// before\nconnector = BlobStorageConnector(bucket_type='s3', bucket_name='docs')\nkeys = list(connector.list_keys())  # raises\n// after\nconnector = BlobStorageConnector(bucket_type='s3', bucket_name='docs')\nconnector.load_credentials({\n    'aws_access_key_id': key,\n    'aws_secret_access_key': secret,\n})\nkeys = list(connector.list_keys())","handlingStrategy":"validation","validationCode":"if connector.s3_client is None:\n    raise RuntimeError('call load_credentials() before list_keys()')\nkeys = list(connector.list_keys())","typeGuard":"def is_connector_ready(c: BlobStorageConnector) -> bool:\n    return c.s3_client is not None","tryCatchPattern":"try:\n    keys = list(connector.list_keys())\nexcept ConnectorMissingCredentialError:\n    # generator: error surfaces at first next(); re-run credential load then retry once\n    connector.load_credentials(creds)\n    keys = list(connector.list_keys())","preventionTips":["Centralize construction: one factory that builds, loads credentials, and validates, used by every entry point","Never swallow ConnectorMissingCredentialError from load_credentials — the connector is unusable afterward"],"tags":["credentials","lifecycle","blob-storage","s3"],"backgroundTag":null,"analyzedSha":"554fb1133ac3861732235ad9c377eb5e0a770665","analyzedAt":"2026-08-15T09:20:16.380Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}