{"record":{"id":"0c9dcbb3a174ee0e","repo":"infiniflow/ragflow","slug":"azure-blob-prune-listing-failed-exc","errorCode":null,"errorMessage":"Azure Blob prune listing failed: {exc}","messagePattern":"Azure Blob prune listing failed: (.+?)","errorType":"validation","errorClass":"UnexpectedValidationError","httpStatus":null,"severity":"error","filePath":"common/data_source/azure_blob_connector.py","lineNumber":259,"sourceCode":"    ) -> Generator[list[SlimDocument], None, None]:\n        \"\"\"Yield batches of slim documents for prune / permission sync.\"\"\"\n        if self._container_client is None:\n            raise ConnectorMissingCredentialError(\"Azure Blob\")\n\n        batch: list[SlimDocument] = []\n        try:\n            for blob_props in self._container_client.list_blobs(name_starts_with=self.prefix or None):\n                name = blob_props.name\n                if not _has_supported_extension(name, self.allow_images):\n                    continue\n                if callback:\n                    callback(name, name)\n                batch.append(SlimDocument(id=name))\n                if len(batch) >= self.batch_size:\n                    yield batch\n                    batch = []\n        except Exception as exc:\n            raise UnexpectedValidationError(f\"Azure Blob prune listing failed: {exc}\") from exc\n\n        if batch:\n            yield batch\n\n    # ------------------------------------------------------------------\n    # Internal document iteration\n    # ------------------------------------------------------------------\n\n    def _iter_documents(\n        self,\n        checkpoint: AzureBlobCheckpoint | None = None,\n        since_epoch: float | None = None,\n        until_epoch: float | None = None,\n    ):\n        from common.data_source.models import Document\n\n        if self._container_client is None:\n            raise ConnectorMissingCredentialError(\"Azure Blob\")","sourceCodeStart":241,"sourceCodeEnd":277,"githubUrl":"https://github.com/infiniflow/ragflow/blob/554fb1133ac3861732235ad9c377eb5e0a770665/common/data_source/azure_blob_connector.py#L241-L277","documentation":"Wraps any exception raised while iterating list_blobs() during the prune/permission-sync listing in retrieve_all_slim_docs_perm_sync. Everything from network resets to SDK deserialization errors becomes UnexpectedValidationError with this prefix and the original exception chained.","triggerScenarios":"Calling retrieve_all_slim_docs_perm_sync and the Azure list_blobs pager throws mid-iteration: transient network failure, SAS token expiring partway through a long listing, or a paged-response deserialization error from an SDK/client version mismatch.","commonSituations":"Large containers where listing outlives a short-lived SAS token, proxies killing long HTTP responses, or intermittent DNS/connectivity issues in self-hosted deployments.","solutions":["Retry the sync job — listings are idempotent, so a simple re-run is safe","For SAS expiry mid-listing: issue a longer-lived token or switch to account-key auth for bulk prune jobs","Check exc.__cause__ to distinguish network errors from SDK errors; upgrade azure-storage-blob if the cause is a parse error","Reduce batch pressure by lowering batch_size so pagination checkpoints are hit sooner"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"# No local pre-check predicts a mid-listing failure; cap exposure by\n# checking credential longevity up front:\nif sas_token and sas_expires_within(sas_token, minutes=30):\n    raise ValueError(\"SAS token too short-lived for a full prune listing\")","typeGuard":null,"tryCatchPattern":"from tenacity import retry, retry_if_exception, stop_after_attempt, wait_exponential\n@retry(\n    retry=retry_if_exception(lambda e: isinstance(e, UnexpectedValidationError)),\n    stop=stop_after_attempt(3),\n    wait=wait_exponential(min=2, max=60),\n)\ndef run_perm_sync(connector, creds):\n    connector.load_credentials(creds)\n    return list(connector.retrieve_all_slim_docs_perm_sync())","preventionTips":["Prune listings are idempotent — make the job retryable by design","Use long-lived or account-key credentials for bulk prune jobs","Log the __cause__ of the wrap so transient vs permanent failures are separable in dashboards"],"tags":["azure","azure-blob","listing","network","transient"],"backgroundTag":null,"analyzedSha":"554fb1133ac3861732235ad9c377eb5e0a770665","analyzedAt":"2026-08-15T09:20:16.380Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}