infiniflow/ragflow · error · ConnectorMissingCredentialError

Blob storage

Error message

Blob storage

What it means

Raised by BlobStorageConnector.list_keys when self.s3_client is None, i.e. load_credentials was never called (or was called but failed before create_s3_client ran). list_keys enumerates the whole bucket keyspace with ETag fingerprints and cannot run without the boto3 client, so the connector treats a missing client as missing credentials. Message is the generic 'Blob storage'.

Source

Thrown at common/data_source/blob_connector.py:192

                if len(batch) == self.batch_size:
                    yield batch
                    batch = []
            except Exception:
                logging.exception(f"Error decoding object {obj.get('Key')}")

        if batch:
            yield batch

    def list_keys(self) -> Iterator[KeyRecord]:
        """Enumerate the full bucket keyspace with per-object fingerprints.

        Cheap path: relies on list_objects_v2 which returns ETag in the listing,
        so no GetObject call is needed. Caches each object's metadata so a
        subsequent get_value(key) call can rebuild the Document without a second
        round-trip to S3.
        """
        if self.s3_client is None:
            raise ConnectorMissingCredentialError("Blob storage")

        all_objects, filename_counts = self._collect_blob_objects(
            start=datetime(1970, 1, 1, tzinfo=timezone.utc),
            end=datetime.now(timezone.utc),
        )
        self._filename_counts = filename_counts
        self._listing_cache = {}

        for obj in all_objects:
            doc_id = f"{self.bucket_type}:{self.bucket_name}:{obj['Key']}"
            self._listing_cache[doc_id] = obj
            yield KeyRecord(
                key=doc_id,
                fingerprint=_normalize_etag(obj.get("ETag")),
            )

    def get_value(self, key: str) -> Document:
        """Materialize the Document for a key previously yielded by list_keys().

View on GitHub (pinned to 554fb1133a)

Solutions

1. Call connector.load_credentials(credentials) and let it complete successfully before list_keys()
2. Ensure the credential dict passes the per-bucket_type validation (R2/S3/GCS/OCI/S3-compatible key sets) so load_credentials reaches create_s3_client
3. Do not catch-and-continue on load_credentials errors in the orchestration code — treat them as fatal for the run
4. In tests, call load_credentials with dummy valid-shaped credentials before exercising list_keys

Example fix

// before
connector = BlobStorageConnector(bucket_type='s3', bucket_name='docs')
keys = list(connector.list_keys())  # raises
// after
connector = BlobStorageConnector(bucket_type='s3', bucket_name='docs')
connector.load_credentials({
    'aws_access_key_id': key,
    'aws_secret_access_key': secret,
})
keys = list(connector.list_keys())

if connector.s3_client is None:
    raise RuntimeError('call load_credentials() before list_keys()')
keys = list(connector.list_keys())

def is_connector_ready(c: BlobStorageConnector) -> bool:
    return c.s3_client is not None

try:
    keys = list(connector.list_keys())
except ConnectorMissingCredentialError:
    # generator: error surfaces at first next(); re-run credential load then retry once
    connector.load_credentials(creds)
    keys = list(connector.list_keys())

When it happens

Trigger: Instantiating BlobStorageConnector and calling list_keys() directly without a prior successful load_credentials(credentials) call; or load_credentials raising ConnectorMissingCredentialError partway so s3_client was never assigned (it is only set at the end of the method).

Common situations: Indexing pipeline orchestrator skips or mishandles the credential-loading step; a failed load_credentials exception is swallowed upstream and the run continues to the fingerprint/listing phase; unit tests constructing the connector without credentials.

Related errors

• S3 Compatible Storage
• Blob storage credentials not loaded.
• Invalid chat_id: ${payload.chat_id}
• main() must be defined or exported.
• Failed to read compilation status

AI-assisted analysis of infiniflow/ragflow@554fb1133a (2026-08-15). Data as JSON: /api/errors/24283e1146c3ffca. Report an issue: GitHub.